If there is nothing else with a gravitational pull, then yes, it would go on forever.
However, this will not work in practice because the galaxy is full of "something else". You could leave Earth's orbit with a velocity of $11.186 km/s$ (that's Earth's escape velocity), but you cannot leave the Solar System, because the Solar System's escape velocity for a spacecraft launched from Earth is $16.6 km/s$. Therefore your spacecraft will eventually turn around (i.e. accelerate).