I understand that the black-and-white images you see looking through a household telescope are only like that due to the intensity of the light that reaches us, and that most of the astronomy images we find online have some color modification. But if we were to eventually travel between galaxies, how would those galaxies appear to the naked eye? How close to the false color images would they be, and how accurately do we know this?
most of the astronomy images we find online have some color modification
How close to the false color images would they be
This is a common misconception, that the pictures you see of galaxies and nebulae are necessarily "false color", "modified color", or "photoshopped". Some of them, yes. But a lot of them are quite simply true color, but taken with a sensor (CMOS, CCD) that doesn't suffer from the limitations of the human eye.
E.g. look at this image of the Horsehead Nebula:
All that color is real. It's there, in the photons reaching you. But your eye cannot see it. A CMOS, however, can. This is not "false color", although saturation was likely increased in post-processing, in addition to what the sensor can do. But the hues are probably real (e.g., the red you see in the image, or the blue, was present in the photons hitting the sensor - albeit at a lower saturation level).
(An astute observer may object that the eye and the CMOS don't see the exact same hues, but let's not go down that rathole now.)
"False color" means when the image shows green where the CMOS (or the human eye, if luminosity was higher) would see red, or something like that. This is not always the case with images of nebulae and galaxies; in fact, if the image was taken with visible light, chances are the hues are preserved.
Proper false color images are those taken in UV or IR, and then artificially converted to visible light. This is an example of it, the Sun in ultraviolet:
Now, to answer your question:
Unfortunately, even from a near distance, most of these objects won't look much better. They are, after all, faint, rarefied clouds of dust and gas. They are just not bright enough for the human eye to see color.
There are few exceptions. A notable one would be close binary systems where the components are stars of very different temperatures. Kind of like Albireo, but much closer. From a starship, looking at the two stars orbiting each other, you'd see very clearly a striking color difference - perhaps a large, somewhat dim, deep red star, and a blinding, crisp dot of bluish white light, the smaller and more active companion.
The views from the center of a globular cluster undergoing a compression phase should be pretty spectacular, too. Night would never be dark on a planet in the middle of the cluster.