I guess spinning in perpendicular direction to main rotor is better in counteracting the torque but what's the mechanism at work?
the thrust force developed by a rotating propeller points along the direction perpendicular to its plane of rotation. Since the objective of the tail rotor is to counteract the torque reaction imposed by the main rotor on the engine which turns it, geometry dictates that the tail rotor has to be perpendicular to the plane of rotation of the main rotor.
Some helos have tail rotors which are canted at a slight angle off perpendicularity, which is a way of accomodating the fact that at cruise conditions (maximum forward airspeed), the airflow over the fuselage and main rotor is slightly asymmetric.
Tail rotors are the obvious solution when you don't want the helicopter's body to start spinning while the rotors remain almost stationary.
Let's imagine that you don't have the tail rotor. According to Newton's third law, there'll be an opposing force when the main rotor spins, and hence the helicopter itself rotates. The tail rotor is positioned a reasonable distance from the helicopter's center of mass, so it provides a torque to allow the helicopter to remain straight. Clearly, this won't be very effective if you had a horizontal tail rotor: the horizontal component of the force exerted by the tail rotor (which is what you need to counter the rotation caused by the main rotor) is directly proportional to the cosine of the angle between the vertical axis and the force exerted by the tail rotor.
Frequently, rotation of the helicopter can be achieved by slowing down or speeding up the tail rotor, though there are better ways (like tilting the main rotor).
More interestingly, the same effect of making sure that the helicopter doesn't start spinning can be achieved with a rotor that's in the same orientation as the main rotor: look at co-axial rotors. Essentially, a coaxial rotor is of the same size as the main one, but it spins in the opposite direction. So it provides lift, but it tries to rotate the helicopter in the opposite direction, which results in no net rotation.
Wikipedia has a great gif to descripe what coaxial rotors are: