I would have been shocked if I had been there as well and it was good that you reacted because the guy would have repeated the same thing if such happened again.
Sometimes, it's good to speak up. It is good that things didn't get out of hand because it could have become a completely different story.