Is it correct to say that the center of mass of any space rocket is stationary here on earth? roughly speaking, without external forces the motion of the center of mass is preserved, so since the rocket is stationary before the start, so is the center of mass still here?
Technically, no, because the initially exhausted fuel interacted with the ground and thus, the system of rocket + fuel was acted upon by the external force. However, once the rocket is at a certain high enough height where you can ignore the interaction of the exhausted gases with the ground, yes, the system of rocket + fuel is not acted upon by any external forces, and thus, its center of motion continues to move with a constant velocity (ignoring the air resistance, and gravity, of course). However, by this time, it would have picked up some velocity, and thus, it won't stay at rest.
In a more idealized version of the scenario, if you imagine a rocket being fired from somewhere in the space where the exhaust gases never experience any force from the ground, you are correct, the center of mass of the rocket + fuel system would remain at rest in the frame of reference in which it was at rest before the launch.
If you are considering just the rocket, then there are external forces acting on it, namely the thrust, drag, and gravity.
If you want to consider an isolated system so there are no external forces, you would have to consider the rocket, fuel, surrounding air, and the Earth.
But yes, you are correct that if there are no external forces then the center of mass will not undergo any acceleration.