How does physics "remember" that there's an EM wave when $E, B = 0$? I don't understand this subject well but I'll try to explain what I mean. In case of a pendulum or other macro oscillators you can calculate the "next" state from position/angle and velocity/angular velocity. They give you the entire description of the oscillator's state. Position and velocity are out of phase - when the displacement is largest, the velocity goes to 0 and vice versa.
In EM waves, $E$ and $B$ are in phase and, if I understand correctly, each of them gives rise to the other. What happens when they both go to $0$? How would physics "remember" that there's a wave? Are there some physical properties like "electric field velocity" or "magnetic field velocity"?
 A: The Maxwell equations contain not only fields, but also their time- and space-derivatives: thus dynamics.
A: A "state" usually involves the time derivative. If you a pendulum at the bottom of its swing, i.e. $\theta = 0$, but if it has some momentum, then $\frac{d}{dt} \theta \neq 0$ and thus in the next time step $t + dt$, it will swing a bit farther.
Now, for a scalar field $\phi$ which satisfies the wave equation, it may be $0$ at some time, but it can have some first time derivative. Then the next moment they won't be $0$ anymore. This is actually what happens in a standing wave (see the gif here). There are moments in time at which the wave is $0$ everywhere. However, a $dt$ later it isn't $0$ anymore.
With the $E$ and $B$ fields, though, there's an extra wrinkle.
Maxwell's equations read
\begin{align}
\nabla \cdot E &= \rho / \epsilon_0 \\
\nabla \cdot B &= 0 \\
\nabla \times E &= - \frac{\partial}{\partial t} B \\
\nabla \times B &= \mu_0 J + \mu_0 \epsilon_0 \frac{\partial}{\partial t} E.
\end{align}
Say we are in vacuum so $\rho = 0$ and $J = 0$. Look at the third and fourth equations. If $E = 0$ then we must have $\frac{\partial}{\partial t} B = 0$. Likewise, if $B = 0$, then we must have $\frac{\partial}{\partial t} E = 0$. Equivalently, if $\frac{\partial}{\partial t} B \neq 0$ then we must have $E \neq 0$ and if $\frac{\partial}{\partial t} E \neq 0$ then we must have $B \neq 0$
So, if one of the $E$ or $B$ fields has a non zero time derivative, then the other field, $B$ or $E$, must have some non-zero value.
If $E = 0$ and $B = 0$ everywhere, then neither $E$ nor $B$ has a first time derivative and thus it will stay $E = 0$ and $B = 0$ forever.
A: This is a long comment:
This is what you are talking about:


Electromagnetic waves can be imagined as a self-propagating transverse oscillating wave of electric and magnetic fields. This 3D animation shows a plane linearly polarized wave propagating from left to right. The electric and magnetic fields in such a wave are in-phase with each other, reaching minima and maxima together.

At the time of the luminiferous ether, before the Michelson Morley experiment set it to rest, the question could be answered in the same way that a plane wave in water would be answered: the wave amplitude is related to the energy of the wave, but the zero of the amplitude does not mean zero in energy as the energy is transported through the medium.
Once there is no medium for the electromagnetic wave, one has to define the energy transport of the wave, and this is done using the average values of the electric or magnetic field of the wave, and the Poynting vector.
Once the existence of photons was established  it is obvious that the energy is carried by the photons, which build up quantum mechanically the mathematical form of E and B fields (see here for example)
A: There are at least two kinds of ways of looking at harmonic oscillators.
One is to consider position and velocity as initial conditions of the oscillator, and acceleration as the dependent quantity, which is determined by the equation of motion (second order differential equation). This is the usual way that a mechanical harmonic oscillator is viewed.
The other way is to consider two independent, equally important state variables (phase space), which are in a sense orthogonal to each other (in a first order differential equation). An easy example for this is centripetal motion, e.g. a weight fixed some distance away from a center point by a thread.
The two views are related to each other in that position and velocity could also be considered completely independent variables (which form the phase space).
A harmonic oscillator in state/phase space (first order differential equation, centripetal motion) form looks like
$$\dot {\vec r}=\frac{d}{dt}\left(x \atop y\right)=\left( \begin{array}{cc} 0 & -\omega \\ \omega & 0 \\ \end{array}\right) \left(x \atop y\right)=\vec\omega \times \vec r$$
(where I have assumed that $\vec \omega$ is a vector along the suppressed z-direction). If you write this component-wise, you get
$$\dot x = -\omega y \qquad , \qquad \dot y = \omega x$$
Substitution of the second into the derived first equation yields after a trivial rearrangement
$$\ddot x +\omega^2 x = 0$$
This is the second order differential equation form of the harmonic oscillator (i.e. the one that is most commonly known by students). You see, both representations are equivalent to each other, although they look pretty different at first.
The point is that Maxwell's equations are of the phase space type (first order differential equations). So the electric and magnetic field are considered independent degrees of freedom, which are kind of orthogonal to each other (of course, the continuous EM field is not a single oscillator, but infinitely many oscillators, but you probably know that). Just similar to centripetal motion, the distinction between position/velocity is taken over by x- and y-coordinates (E- and B-field).
If you want to cast Maxwell's equations into the second order differential equation form (and hence, develop some notion of "position"/"velocity"), you either have to derive Maxwell's equations (like I did for centripetal motion above) and obtain the wave equations, for example for the electric field in vacuum
$$\frac{1}{c^2}\frac{\partial^2 E}{\partial t^2}+\frac{\partial^2 E}{\partial x^2}=0,$$
or you have to introduce the electromagnetic potentials $\vec A$ and $\Phi$, which serves as the "position", while their derivatives (the fields) represent the velocities. In Lorentz gauge, the free wave equations then become
$$\frac{1}{c^2}\frac{\partial^2 \Phi}{\partial t^2}+\frac{\partial^2 \Phi}{\partial x^2}=0$$
$$\frac{1}{c^2}\frac{\partial^2 \vec A}{\partial t^2}+\frac{\partial^2 \vec A}{\partial x^2}=0$$
For a certain harmonic plane wave (i.e. one "single oscillator" of the wave) the spatial derivative reduces to something proportional to the square of the wave number, e.g. for the E-field:
$$\frac{\partial^2 E}{\partial x^2}=-k_x^2E$$
and by the dispersion relation $k_x^2c^2=\omega^2$ we finally get the harmonic oscillator in second order form again:
$$\frac{\partial^2 E}{\partial t^2}+\omega^2 E=0$$
Similar relations hold for the magnetic field in vacuum. Just like for any harmonic oscillator, velocity (first temporal derivative, $\dot E$) does not appear in the differential equation, but is part of the initial conditions.
Neither way of viewing the harmonic oscillator is more "valid" than the other. But the chosen view limits what can be considered position/velocity.
In order to return to the core of your question: if E and B are equal to zero for a single plane wave oscillator (one specific wave number vector $\vec k$ and polarization, also sometimes called a "mode"), this means that they are zero for all time for that oscillator, because "in state space form a zero initial state also means zero evolution". Physics does not remember that there is a wave, just because there is no wave at all due to the initial conditions.
A: For the photon in a box with the right distance between the walls as a multiple of the wavelength, the E and B fields are out of phase and a sketch looks like this:

Here it is obvious that the photon has a constant energy content. Which also corresponds to our intuition.
