the coordinate position of a test mass does not change in the TT-frame if the test particle was at rest
The test mass being at rest in the TT frame is an assumption. The conclusion is only valid under that condition. If the test mass were moving in the TT frame, it would continue moving as the GWs passed. The test mass's coordinate velocity would likely change due to the GWs (depending on polarization and direction of motion and the like).
The question seems to be: is this a realistic assumption about the universe?
The thing you are worrying about seems to be that if the GWs existed far into the past, then the test mass must have been at rest in the TT frame far into the past too. It must not have interacted with anything else either (GW calculations typically assume a small perturbation on a vacuum, Minkowski spacetime).
Whether these are valid assumptions has to do with the relative scales of the physical effects. For short duration transient GWs, we can assume the test masses in a GW detector like LIGO are approximately in free fall and approximately at rest in the TT frame. The test masses are not in true free fall, nor are they at rest. The motion of the Earth is slow compared to the scale of the GWs, so it doesn't matter so much. For a long lived, continuous GW source, the data analysis has to take into account the motion of the detector (and the source). The introductory "Chapter 1" presentation will have to be modified to deal with those kinds of sources.
If you want to think about a stochastic background of GWs, it helps to work out the idealized case first. Imagine some test masses at rest in the TT frame and figure out what happens. If you want to build a GW detector and analyze its data, you will have to worry about some extra complications.