Galaxies 4,200 megaparsecs away will travel faster than the speed of light away from each other, so why don't we see those galaxies as Going back in time up to the moment of the big bang?
There is a common misunderstanding in your question, that is based on the fact, that you would think that if you travel faster then light (in your case the galaxies), then you go back in time.
Now galaxies are receding faster then light because the space inbetween is expanding faster then light, and not because they would be traveling faster then light.
So there is no contradiction with SR, which states that nothing can travel faster then light. Nothing is traveling faster then light. It is space itself that is expanding inbetween the galaxies, so there is no contradiction with SR.
Since nothing is traveling faster then light, nothing is moving back in time (even that would be just a theory).
To see that this is impossible, imagine a series of photons emitted in regular succession from the distant galaxy. They follow each other in a line from that galaxy and head towards our own. No one of these photons can pass any other photon. To do so, one of them would have to be travelling faster than the one it was passing. But this is clearly absurd: all photons travel at $c$. Therefore, all the photons arrive in the same order they left in. There is no illusion of time travel.