I'm designing a space ship for a comic I'm writing. I was just finishing up the first sketch when I thought whether such a design would actually be possible. Hence my question:
A spaceship is located in deep space. A propulsion system (let's make it a rocket) is attached to the side, and thus not in one line with the ship's center. Would this spaceship travel in a straight line, or rotate?
This is superficially similar to the question If I push or hit an object in space will it rotate or move along a straight line? However, in that question the force is either impulsive (very short duration), or continues to act in a fixed direction in space. The result is that the centre of mass (CM) of the body moves along a straight line while the body rotates about its CM. If the force continues to act (along the same line) then there is both linear and rotational acceleration.
The complication in your question is that the direction of the force rotates along with the space ship. This makes the resulting motion more complicated.
Your question has been asked in Force applied off center on an object and equations of motion given, but the kind of motion which can be expected has not been described.
If the force $F$ is applied at perpendicular distance $r$ from the CM, in a direction which is fixed within the body, then there is a constant torque $rF$ which produces a constant angular acceleration $\alpha = rF/Jr$ about the CM, where $J$ is the moment of inertia about the CM.
At the same time there is a variable linear acceleration $\vec{a}=\vec{F}/M$ of the CM in the instantaneous direction of the force. This direction is constantly changing, with an increasing rate of change of direction, so the net acceleration over a complete rotation tends toward zero, leaving a constant velocity.
To summarise :
The resulting motion is a combination of (1) constant angular acceleration about the CM, and (2) after a long time, constant velocity of the CM in a fixed direction, with initial oscillations about this direction.
