Torque $\tau$ causes angular momentum $L$ in a linear manner, $\tau=dL/dt$, so torque and angular momentum are in the same direction (lets call them parallel). Torque itself comes from a force $F$ acting at a distance $r$ from the rotation centre:
This is a cross-product. A cross-product mathematically gives a resulting vector which is perpendicular to both of the involved vectors. So, the resulting $\tau$ vector is perpendicular to both the $F$ vector and the $r$ vector. This is simply a mathematical quirk due to the cross-product operation.
Now, it is not that simple to talk about the "direction" of a rotational parameter. What is the direction of angular velocity or angular momentum or angular acceleration? You would maybe say clockwise or counter-clockwise, but that is a curved direction and not easily expressed mathematically.
Instead, it is easier mathematically to talk about the axis, about which the rotational parameter acts. When a torque turns a wheel, it is turning about an axis through its centre. So, if you can define that axis, then you have a hold on the direction, mathematically (which can be described with coordinates and such - much easier to work with).
The torque vector found in the cross-product thus points along this rotation axis. And then we use the right-hand-rule to say that if the axis points outwards (out of the screen/paper), then the actual direction is counter-clockwise and vice versa.
This is why you often in technical literature will hear the description of "a direction along an axis", even though we are talking about a rotational parameter. Saying "directed along an axis" actually rather means "turning about that axis".