Honestly, it was very annoying, he probably transferred his agression to me and it wasn't nice at all. Considering his kind of job, he could have handled things better.

This further maths teacher of yours sounds like a whole lot, good thing he liked you. Maybe they should offer anger management programs in different workplaces. Imagine if he was teaching younger students, he could land himself in some big trouble if he doesn't know how to control his anger.