Why does it accelerate linearly, if the only force I'm applying is a torque?
A torque $τ$ is simply the moment about a point created by the product of a force $F$ times the distance $r$ between the point and the force perpendicular to the moment arm. So a torque cannot exist without one or more forces.
To analyze the motion of the body subjected to forces with respect to a point in space you need to take the sum of the moments about the point, and the sum of the forces. If the sum of the moments and forces are zero, the body is in equilibrium.
If there is a net force applied to a rigid body, no mater where it is applied, it will experience a linear acceleration (translation). If that net force is not applied to the center of mass (COM) of the body, the body will experience, in addition to the linear acceleration, an angular acceleration (rotation) due to the moment (torque) about the COM.
In order for the body to experience rotation without translation, you would need to apply what is called a “couple”. A couple is a system of two parallel forces that are equal in magnitude, and opposite in direction. The rotation occurs about a point midway between the parallel forces. Consequently you have a zero net force and there would rotation without translation.
The linear and angular accelerations described above will, of course, last only as long as the forces and torques are applied to the rigid body.
Hope this helps.