When studying angular things - torque, angular velocity, angular momentum, etc. - physicists do a clever thing to avoid having to describe curves. You see, you might be tempted to draw a curved arrow for a torque, indicating that you are twisting something around in a circular-ish way. But then when you try to add two such arrows together, all of a sudden you realize your notation no longer has a natural, intuitive meaning.
Instead, we draw the arrow pointing perpendicular to the plain of the curve you are tempted to draw. More precisely in the case of torque, perpendicular to the plain defined by the radial vector and the force vector. Note that this uniquely defines what plane your curved arrow must reside in, and, given the right-hand rule, clears up the ambiguity as to which way your curved arrow should point (if your right-hand fingers curl in the direction of the curved arrow you want to draw, your thumb points in the direction of the straight arrow you should draw instead).
It is then a simple matter to encode the magnitude of the torque/angular velocity/whatever in the length of this vector. The benefit is that you end up with straight arrows describing everything, and they add exactly as your torques should add - you have a genuine vector space, and are free to abstract away from all diagrams. And it is not even terribly counterintuitive - the torque vector is parallel to the axis around which you are applying torque. If you think about it long enough, you should be able to convince yourself that if you had to choose a single direction to define things, this is the least ambiguous.