How is momentum conserved in this situation? Let’s say we have two inactive electromagnets spaced one light-minute apart, with north poles pointing toward each other. One of the electromagnets is turned on for ten seconds, and then turned off again (or destroyed). Once the magnetic field from the first electromagnet has propagated to the second, the second electromagnet is turned on. Would the second electromagnet not be repelled away by the magnetic field without exerting a force on the first electromagnet? How is momentum conserved in this scenario? I think the same question could be asked about electric fields and dipoles in place of magnetic fields and electromagnets.
 A: The turned off field will be a very long wavelength electromagnetic field, with its momentum, which will hit the magnet and transfer its momentum to the solid state lattice of  the magnet.  The second magnet will send into space, the field meeting nothing on its way .
Edit after comment:
The basic frame of all theories in physics, mainstream at present, is quantum mechanical, and the mainstream accepts that the standard model elementary particles are the basic constituents
In this model the static electric and magnetic fields are built up from very low energy photons, the basic constituents of light and all electromagnetic waves. When your electromagnet is turned off, a pulse is created traveling in vacuum of very long wavelength photons which carry momentum and energy and transfer it by collision to the live electromagnet.
Alternatively, within classical electrodynamics, again a pulse of light is generated (because changing magnetic fields generate electric fields and elecromagnetic radiation according to Maxwell's equations), and this pulse will have energy and momentum according to the classical model.
(Permanent magnets cannot be turned off. The energy of repulsion or attraction between permanent magnets comes from the tiny magnetic dipoles that make up the field, they get reoriented and the magnets are a tiny bit demagnetized.)
