A guy started on me...?

You did the right thing. If you would stayed to fight 3, then you, and possibly the girls, would have been hurt. Wait until later, and get 5 of your friends to go back with you, and hunt him down.