A force is required to change the momentum of the rocket exhaust which consists mainly gas moving out of the rocket chamber. People think that it is the rocket that pushes the gas out. If this were true, yes the gas would push back. However, the gas moves because of pressure gradient force and not because of the rocket pushes it out. Therefore, there would be no push on a rocket.
https://en.m.wikipedia.org/wiki/Pressure-gradient_force
When the gas moves out at high velocity, the atmosphere provides resistance which creates back pressure of flow causing thrust. In space, there is no such resistance.