If we can see a galaxy can that galaxy see us? This is a question about the properties of the expansion of the universe. I can't say it any better than: If we observe a primordial galaxy that existed soon after the Big Bang, does it follow that the same galaxy, at roughly the same number of years after the big as we are today, can observe our galaxy as it was soon after the Big Bang. I only mean theoretically. Please ignore the galaxy merger issue and issue of new stars since that time, if the question can be constructed logically. 
 A: Yes.
The Milky Way is a very old galaxy, having formed roughly half a billion years after the Big Bang. So if we observe a galaxy that has a redshift of ~10, we are looking back in time to approximately this epoch, so an alien astronomer in that galaxy observing the Milky Way today  would see it redshifted by the same factor, and would observe it in the process of forming.
If we observe a galaxy at redshift ~0.5, we are looking 5 billion years back in time, so an alien astronomer in that galaxy would see the Milky Way as it looked when it was roughly 8 billion years old, and with a magically powerful telescope. it would be able to see our Sun in the process of forming.
This does not really have anything to do with time symmetry. The light that leaves the distant galaxy and the light that leaves the Milky Way at the same time in each other's directions simply travel through space, meet each other halfway (without interacting) and reaches the other galaxy at the same time.
A: If the Maxwell equations are valid up to the Galaxy of which you talk, and all the time, then the answer is "yes", because these equation are symmetric in time. That means, the trip of the light from the galaxy to us, is reversible in time, i.e. occurs also in reverse.
It's simple to show that the time symmetry is equivalent with rolling the movie in inverse order, i.e. the path of the light is the same if we take the target for source and the source for target. This is one of the laws of the geometrical optics.
EDIT: About the fact that each one of us, the observers in the galaxy, and the observer in our galaxy, would see the past of the other, it is true.
A: Generally speaking yes.
But looking in detail not necessarily so...
Basically, observers today can play peekaboo with the cosmic event horizon by running away from the primordial object fast enough.
Imagine being in a fast spinning binary, as you approach your object of desire everything is fine and blueshifted. As you recede everything is redshifted, very very very redshift.... oops where did that superluminal galaxy just go to ? Oh, there it is again as I reach aphelion.
Coda: this is in a standard cosmology, which I do not believe to be an accurate representation.
