In short, you are to think of the direction of the torque as pointing along the axis of the rotation it would induce in a rigid body initially at rest.
But if the conception of torque as a vector out of the page seems artificial, that's because it is.
Torque is not fundamentally a quantity that is a vector but a directed plane or directed area. Such an object is called a bivector. When we speak of torque as a vector we're using a non-general definition that works because we happen to live in three dimensions.
Rotations fundamentally transform planes, rather than leaving axes invariant. In a general number of dimensions, you specify a rotation (more often known as a proper orthogonal transformation in such a context) by specifying its action on two-dimensional, linearly independent subspaces. The simplest possible rotation acts on one plane, but orthogonal transformations can act on many planar subspaces at once.
Torques, producing rotations, are also, fundamentally, bivectors.
In three dimensions, there is at most one plane that can be acted on in this way, so we can cheat a bit and define the rotation through its axis, because in three dimensions because one can define a plane by the unit normal vector to it. But in four dimensions, you cannot define a plane by a unit normal vector. The orthogonal complement of a plane in four dimensions is another plane, so that the axis notion in four dimensions is meaningless - it won't define a rotation and could not define a torque if we lived in a four spatial dimensional universe and had occasion to calculate moments of forces there.
Another notion you may meet in the future is the Hodge Dual. This is a generalization of the defining a plane through an axis in three dimensions, as is done with torques and rotations.