Yes, electromagnetism without sources (i.e. charges) is scale-invariant.
This can also be seen from the fact that if a plane wave of wavenumber $k$ and angular frequency $\omega$ is a solution, then so is a plane wave of wavenumber $ak$ and frequency $a\omega$, for any constant real number $a$. In one spatial dimension, $\exp[i(kx-\omega t)]$ is a solution for any $\omega$ and $k$ such that $\omega/k=1$ (i.e., the speed of light), where we are using $c=1$. And scale symmetry then simply says that any such wave is a solution as long as $\omega/k=1$, since $a\omega/ak = 1$ also. And since that is so the same is true of any linear superposition of plane waves, each traveling at light speed. Thus we can form arbitrary waves (with some reasonable mathematical restrictions), and as long as the component plane waves have light speed, the arbitrary wave will also have light speed and satisfy the electric or magnetic field equations (or electromagnetic waves, with $\mathbf E$ and $\mathbf B$ perpendicular, which needs more than those two equations.
There is no scale because the scale for the spatial component and the time component need to be the same when the only restriction is the light speed. For sources it can now be different: a certain length dipole will be a source, or sink, for electromagnetic fields whose spectrum is preferentially peaked at lengths related to the antenna length and the related frequencies and their harmonics. We have introduced a length scale. Most or all sources will introduce length and/or time scales.
As the Wikipedia article referenced states, and easily shows, QED without charges (i.e., sources or sinks) is also scale-invariant. That is true because the photon is massless. In a quantum field theory without mass the source-less field is scale-invariant. Its radiation scales as $1/r$ for the field (i.e, the amplitude). The linear approximation to classical General Relativity also has its waves travel at $c$ and go like $1/r$ in amplitude. The quantum of the field is the still undetected graviton (though an accepted quantum gravity theory still does not exist.
Similarly for other quantum fields, if they are massless they would be scale-invariant; an example is the scalar massless Klein Gordon field. As soon as a mass term is introduced with the introduction of an $m^2$ in the equation, it no longer is scale-invariant, and the field diminishes exponentially with $m$ as the scale.