More specifically, the final-state particles generally are moving away from each other, and so are unlikely to collide, whereas the initial-state particles generally are moving toward each other, so they are likely to collide.
Note that there are, in fact, situations in which you do see the final-state particles return to the initial state. One great example is positronium production and annihilation. A photon of energy greater than $2m_e$ can produce an electron-positron pair, and another photon is required to interact with the electron or position to obey energy and momentum conservation. So the reaction here is $\gamma+\gamma\to e^++e^-$. If the photon's energy is only slightly greater than $2m_e$, the electron-positron pair will become bound in a positronium atom. Eventually, this positronium atom annihilates, turning back into two photons (sometimes three, but for our purposes we only care about the two-photon decay), so we have $\gamma+\gamma\to e^++e^-\to\gamma+\gamma$. If you wait more than a few microseconds after observing the initial state, then you'll never see the two-electron final state. In this instance, the kinematics are compatible with a return to the initial state, since the "final-state" electrons don't move away from each other, but instead remain bound near each other.