Why do EM waves propagate forward in time rather than backward in time? Why do EM propagate forward in time, but not backward in time? Stated differently, why does EM radiation not "ripple inwards" and collect at some point? These are perfectly possible, by time-reversal.
I'm assuming the explanation will invoke entropy somehow, but it's not yet clear to me how, as we explain similar problems in Stat Mech that way. Maybe it's baked into the initial conditions of the universe (as our Stat Mech explanations require)?
EDIT: I've thought about it some more, and I think my question is this one: why in Quantum Mechanics are we free to use the advanced Greene's Function in place of the retarded one, but not here? (Are we?) Maybe the advanced/retarded potentials result in different physically arrangements of the fields, while the advanced/retarded Greene's functions don't result in physically different wavefunctions? I think that answering this would resolve my question. (Insofar as I would know why we throw out advanced potentials, but certainly it would be unresolved why this area in particular violates time-symmetry. Isn't time-symmetry a major principle? Why would we say the universe is time-symmetric if it isn't in E+M?)
 A: The reason is that time-reversal symmetry is already broken in the way we perceive time.
We see time as something that flows from the past to the future, and we try to deduce the future from the past. Hence, we consider electromagnetic wave propagation (or time evolution of any dynamical system) as an initial-value problem, as opposed to a final-value problem. It is just that retarded solutions (outgoing waves) are what arise when solving initial-value problems.
To see how this is so, suppose there is a current source with a finite duration in time and an EM wave $\textbf{E}(\textbf{r},t)$ consistent with this source distribution. Naturally, we decompose the wave into 
\begin{equation}
\textbf{E}(\textbf{r},t) = \textbf{E}_{0}(\textbf{r},t) + \textbf{E}_{\rm{source}}(\textbf{r},t),
\end{equation}
where $\textbf{E}_{0}(\textbf{r},t)$ is a homogeneous solution of Maxwell's equations, and $\textbf{E}_{\rm{source}}(\textbf{r},t)$ is nonzero only after the source is turned on. Such $\textbf{E}_{\rm{source}}(\textbf{r},t)$ is what we call the retarded solution; it corresponds to an initial condition such that there exists no field before the source is turned on, and we adjust $\textbf{E}_{0}(\textbf{r},t)$ when a different initial condition needs to be satisfied. A key characteristic of a retarded solution is that its space-time dependence corresponds to an outgoing wave from the source.
If, for some reason, we study a final-value problem, we may consider a field decomposition just like the above except that now, $\textbf{E}_{\rm{source}}(\textbf{r},t)$ is nonzero only before the source is turned off. In this case, $\textbf{E}_{\rm{source}}(\textbf{r},t)$ is an advanced solution, and it corresponds to a final condition such that there exists no field after the source is turned off. Also, it is what we get by reversing the time from the retarded solution, i.e., a wave incoming from spatial infinity.
In summary, we only see electromagnetic waves that propagate forward in time (i.e., retarded solutions) because we (knowingly or unknowingly) consider initial-value problems. Time-reversal symmetry manifests itself in the fact that if we reverse the direction of time, a solution to an initial-value problem of Maxwell's equations is transformed into a solution to a final-value problem. Of course, the symmetry appears to be broken if we look at initial-value problems only; that is, we only see retarded solutions.
A: 
Stated differently, why does EM radiation not "ripple inwards" and collect at some point? These are perfectly possible, by time-reversal.

Well, in most cases observed or measured, EM radiation does not ripple outwards from a point either; usually, the radiation is connected to a body with non-zero spatial dimensions. In fact, radiation of point charges may contain advanced field component but macroscopically look like the retarded solution - see the Feynman-Wheeler theory.
I suppose you are interested in the following question:
since there are well known processes in which EM waves are created in and propagate outwards from physical bodies out to distant space (such as radiating antenna), why are there not also inverse processes, where the EM waves propagate from the distant space towards some physical bodies and collapse on them?
We know that Maxwell's equations in some simple scenarios (such as continuous motion of a point charge) have solutions like that (called advanced fields as opposed to retarded fields), so why aren't we observing such collapsing spherical waves at least somewhere?
Short answer: the reason can be sought either in:


*

*the state of the EM field in the past (a special initial condition that does not lead to macroscopic advanced waves); past is not a part of physical laws, it must be assumed as a separate assumption;


or


*

*it can be sought in a separate physical law that restricts the fields of charged particles to be the retarded solutions of field equations with their individual source terms.


Long attempt at an answer:
If we knew the state of the field everywhere at some point of time and if we knew the subsequent motion of the charges, we could predict the field into the future. If we fix the motion of the charges, the field would be determined by the initial condition.
The initial condition could be, in principle, such that field in vicinity of charged bodies would evolve like an ingoing wave, coming from far away and collapsing on those bodies. 
Another initial condition could be such that the opposite would happen; the field would evolve like an outgoing wave, getting away from the bodies to infinity.
There are infinity of other initial conditions which are different from the above; in general, the difference between them is a "free field" - a solution of Maxwell's equations without any sources.
The appropriateness of initial condition in macroscopic theory is to be judged based on experience; there is no physical law that would require one or the other.
Absence of ingoing EM waves in our experience means that certain class of initial conditions for fields is to be avoided or even rejected in macroscopic EM theory.
But this does not easily translate into microscopic theory. Let us assume that total EM field is composed of elementary EM fields of very big number of charged particles.
It is possible to have macroscopically retarded fields that are made of microscopic fields which are not purely retarded but contain advanced field. And it is possible to simulate macroscopic advanced field with a special arrangement of microscopic fields that are purely retarded.
An interesting but not the only possible version of EM theory is that the microscopic fields are completely symmetrical combination of retarded and advanced waves (Tetrode, Frenkel or Feynman-Wheeler models and their variations), but when these are used to explain our experience with macroscopic bodies, things are not so simple anymore: if the elementary fields are symmetrical combination of retarded and advanced waves, how come we do not see such symmetrical EM wave around an antenna? (People actually proposed an experiment and verified that the EM field is not such symmetrical one. Experience suggests that EM field near antenna is well given by the retarded solution.)
In the Feynman-Wheeler model, they came up with an interesting idea: they introduced a boundary condition into the theory (so-called absorber condition) which formally allowed them to arrive at realistic description of EM field where the observable field seems to be almost retarded and where the correction supposedly explains radiation reaction. I find the absorber condition very formal and unnatural, and their explanation of radiation reaction (which relies on that condition) as ill motivated and unnecessary for point particles. Still, the idea of symmetrical half-retarded, half-advanced field has some interesting implications, for example, systems of opposite charges are much more stable, because now the waves go both in and out and there is no intense radiation of EM energy to surrounding space from such systems as would be expected based on Larmor's formula (which is not valid here). It is possible that the symmetrical microscopical fields with the right probabilistic assumptions can be consistent with our macroscopic experience with retarded fields, even if we abandon the somehow unnatural absorber condition.
The most simple and natural stance currently is that the elementary fields are retarded and the advanced solutions are unphysical - the second variant from the short answer. This simple choice is quite intuitive - elementary waves only ever propagate outwards from the particles. It explains why macroscopic waves are retarded without further assumptions such as the Feynman-Wheeler absorber condition. True, a collapsing approximately spherical wave could be created if lots of particles danced in a special way, but such correlated motion across great distances is quite improbable, so this poses no challenge for the model.
A: This problem has not really impinged on my attention, but I think you are correct that entropy can explain it, when expressed as a number of microstates.
I quote from the relevant conclusion of the answer by Ján Lalinský :

Collapsing spherical waves can still exist, but only as a special situation involving specially correlated particle motions in the past, far from the center.

For simplicity take a single source. The electromagnetic field emerges from a confluence of a large number of photons. Conceptually the wave functions of the individual photons in superposition build up the E and B fields of Maxwell's equations.  The photons,  as elementary particles are countable. As they spread in the universe the number of microstates increases or remains the same , unless it spreads in completely empty space , which classically is possible , but quantum mechanically the photons will meet vacuum fluctuations, this will involve increasing numbers of microstates.
In order for an advanced wave to reproduce and focus on the same spot, in fact make an inverse source, the photons  must interact with the same order of  vacuum fluctuations, because the photons will travel the same distance to end up at the same (x,y,z). It will have accumulated entropy of the order of the entropy acquired by the outgoing  retarded wave . But the redarded waves source at an (x,y,z) is a at a lower entropy at the start of radiation, the reurned has gathered all the microstates on the way into its entropy, so there has not really been a time reversal to the original solution.
I think this is a reductio ad absurdum argument, which might be combed to be presentable.
A: It's counterinuitive, but I think the question

Stated differently, why does EM radiation not "ripple inwards" and collect at some point?

can be answered thusly:

It does.

Or rather, one could choose to see the same system in terms of 'rippling outward', or in terms of 'rippling inward', without changing the underlying physics. In other words, the question

why in Quantum Mechanics are we free to use the advanced Greene's Function in place of the retarded one, but not here? (Are we?)

Can be answered:

We are.

For example, imagine an idealised model of a point source of radiation surrounded at some finite distance by absorbing walls. The obvious description is in terms of retarded waves radiating outwards from the point source, but if one were so inclined one could instead see the same solution as a sum of infinitely many advanced waves converging onto each point on the absorbing boundary. Admittedly I haven't done a calculation to check, but I think it should be possible to do this such that the sum of all these advanced waves gives exactly the same solution as the single retarded wave.
Of course, in this example and many others it is much more convenient to think in terms of retarded waves than advanced ones, and there is also an important question about why this should be the case. This I think has to do with causality. In the example above, if we move the point source we will change the whole radiation field, but if we move the boundary around we will not. We're able to directly manipulate the sources of radiation, but we can only manipulate radiation sinks indirectly, by manipulating the sources.
This seems slightly strange and mysterious in the context of electromagnetic radiation, but it's not really any different from any other physical process --- quite universally, we can only manipulate initial conditions and not final ones. (As an aside, it is possible to derive the second law from more or less this fact alone.{1})
I came to these conclusions while reading {2}, which covers this issue in some depth. However, I read it a long time ago and can't remember if my conclusion is the same one the author comes to.
{1} Jaynes, E. T., 1965, `Gibbs vs Boltzmann Entropies,' Am. J. Phys., 33, 391; 
{2} Huw Price Time's Arrow and Archimedes' Point, Oxford University Press, 1996
A: Not sure anyone answered the question. The reason being their concept of time. There is no such place as the past nor the future. There is only the present. Einstein developed some interesting theories-he did so in 'his' present. He tried to tell everyone time depends upon your frame of reference. Neil Armstrong walked on the moon in 'his' present time. A photon has momentum but no mass. Assume you are a photon 13 billion miles away. It would take 13 billion years to get here. However, if you were that photon then the trip would be instant(frame of reference). Someday man may walk on the plains of Mars. They can only do so in their present.There has never been an instance where [anything]has gone back in time. 
A: For fundamental questions you must not only limit to a consideration of coordinate time, but you also have to take into account proper time which is the fundamental concept which is underlying the concept of coordinate time of spacetime.
The proper time of lightlike phenomena such as EM waves is zero. That means that lightlike phenomena are time-symmetric, there is no intrinsic time direction. By consequence, any definition of a time direction is stemming from the observer who is observing the EM wave moving at velocity c forward in time. 
That means simply that lightlike EM waves are interacting with their observer in the time direction of the observer. Based on this conclusion we could speculate that - if there are objects moving in the opposite time direction - that these objects would observe EM waves moving in the opposite time direction.
