I think, you mix two or even three kinds of redshift effects.
First, the classical Doppler shift due to which light emitted from a moving object is shifted to the red or blue. This is dependent on relative movement of sender and receiver only. For transverse motion we still get a redshift (see Wikipedia URL below), but it is very small for low speeds.
Second, there is the cosmological redshift which occurs in an expanding universe (or a blueshift in a contracting one). This is only dependent on the universe's sizes at the times of emitting and receiving the light and not affected by any kind of peculiar speeds, neither parallel nor radial. However, if a galaxy is very near to the Milky Way, it is not following the Hubble flow (the expansion of the universe) but rather stays gravitationally bound to it, so there is no cosmological redshift.
Third, the gravitational redshift which occurs if the photon is escaping a gravitational potential. This is only dependent on the source's mass (or whatever mass is creating a potential well in the emission point. If the universe is perfectly homogeneous, then there are no potential wells and no gravitational redshift. In reality it's not, but I guess, this is not the effect you are interested in.
I think, this table summarizes pretty much what I wrote so far: http://en.wikipedia.org/wiki/Redshift#Redshift_formulae
If this doesn't answer your question, please specify the conditions more precisely so that it is clear which types of redshift apply in which way.