You are confused because you are not taking the relativity of simultaneity into account in your analysis. When you have two mirrors moving to the right with velocity v, you are thinking that it takes light a long time to go to the right, but a very short time to go to the left, because in one direction, the mirror is going along with the light, and in the other direction, the mirror is going toward the light.

You conclude that the fellow who is riding along with the two mirrors can't see the same time for the forward leg of the light circuit between the two mirrors as for the backward leg. This conclusion is wrong, because the fellow riding along with the two mirrors has a failure of simultaneity--- the notion of "right now" is altered, so that the "right now" moment at a distance x along the direction of motion is further into the future by vx.

This asymmetrical simultaneity failure means that as the light is going to the right, it is going into the future of the moving observer slower than when it is going to the left, so that from the point of view o the moving observer, both the forward motion and the backward motion of the light go an equal number of time steps into the future, relative to the notion of simultaneity appropriate for the moving observer.

This is explained with a diagram in this answer: https://physics.stackexchange.com/questions/12435/einsteins-postulates-minkowski-space-in-laymans-terms/13621#13621

If you understand why the line of simultaneous events for the moving observer slopes up as viewed by the stationary observer, you will immediately understand why the forward and backward motions are symmetric for the moving observer. This is explained in the first and second diagram of the linked answer, which explicitly use right-moving and left-moving light to establish the slope of the simultaneity line.