When does the concept of electric field in classical electrodynamics fail, and QED is needed? It is really hard to find reference to when the traditional concept of electric wave, especially TEM wave, fails, and needs to be replaced by quantum electrodynamics. 
So when does the concept fail? At high frequencies of electric field?
 A: .Your title asks about the electric field. The content is about the electromagnetic waves, two different entities.
Electromagnetic waves emerge from an innumerable number of single photons. As one cannot have water waves with just a few molecules but need of the order of $10^{23}$ (avogadros number) one cannot measure electromagnetic waves if the photons are few .
The electric field depends  on a large number of charged molecules . If that number is small it will end up with the order of magnitude of the electron charge, a small number , and special scattering experiments in vacuum will be needed to detect it.
As a rule of thumb, the interface between a successful classical description and the need for a quantum mechanical one depends on $\hbar$, a very small number. When the dimensions of the study can assume it as zero than the classical regime is fine. If the values measured are commensurate with the various Heisenberg uncertainty equations, then quantum mechanical estimates are necessary. 
A: It fails when the photon number is small. Since the electromagnetic field can never be zero because of the third law of thermodynamics this automatically couples the temperature, the effective volume and the photon number to each other. As a result it is experimentally impossible to do experiments with single photons at low frequencies because we can't produce the required ultra-low temperatures to make the photon number small.  
A: The failure of classical electrodynamics has to occur when we are talking about very low intensity light (such the wave is describing one photon) or when the frequency of the classical waves is very high (because a photon can decay to an electron-positron pair but an EM wave cannot).
A: The problem with all the answers posted so far is that they are inconsistent with the historical narrative. There were three "nails in the coffin" of classical electromagnetism: namely,


*

*The black body spectrum

*The photo-electric effect

*The Compton effect.


All of the critical experiments measuring these phenomena were done at the macroscopic level, with millions of photons. Not one or two at a time. Millions of them. And the theoretical calculations which purported to debunk the wave theory had nothing to do with assuming very low intensities.
So it is peculiar that all the respondents who have posted so far claim that the critical failure of classic e-m occurs only when intensities are very low.
A: Just to add that an EM field could be fully quantum mechanical with any number of photons (small, medium or even large), when the uncertainty in the number $n$ is very small ; $\Delta n \ll 1$.  This way, the wave amplitude is well defined, but then the phase is very uncertain : $\Delta \phi \gg 1$ :
\begin{equation}\tag{1}
\Delta n \, \Delta\phi \gtrsim 1.
\end{equation}
The classical wave description is valid when you can minimize both $\Delta n$ and $\Delta \phi$.
For example, this quantum state may have an arbitrary large value of $n$ :
\begin{equation}\tag{2}
| \psi \rangle = | n \rangle.
\end{equation}
In this case, we have $\Delta n = 0$ (for any value of $n$), and yet it's a full quantum state.  The wave phase $\phi$ is not defined in this case.  We could also have this Glauber quantum state :
\begin{equation}\tag{3}
| \psi \rangle = \sum_n c_n \, | n \rangle = \sum_{n \,=\,0}^{\infty} \frac{1}{\sqrt{n!}} \, \lambda^n \, e^{- \langle n \rangle/2} \, | n \rangle.
\end{equation}
which gives a Poisson distribution of $n$ with average $\langle n \rangle = \sqrt{|\lambda|}$.  In this case, the phase of the wave is defined from the angle of $\lambda = |\lambda| \, e^{i \phi}$.
For more details, see this Wikipedia page : https://en.wikipedia.org/wiki/Coherent_states
