Problem with the vacuum energy upon quantization of the EM field in Lorenz gauge

Question

There seems to be an issue arising concerning the vacuum energy of the EM field after quantizing in Lorenz gauge.

As usual, we consider an extra gauge-fixing term in the Lagrangian of the form $-\frac{1}{2\zeta}(\partial_\sigma A^\sigma)^2$, so that

$$\mathcal{L} = -{1\over4}F_{\mu\nu}F^{\mu\nu} - \frac{1}{2\zeta}(\partial_\sigma A^\sigma)^2\tag{1}$$

(Natural units are used). In the Feynman-'t Hooft gauge, $\zeta=1$. Then we can quantize the vector potential $A_{\mu}$ canonically by expressing it as follows,

$$A_{\mu}(\vec x)=\int\frac{d^3p}{(2\pi)^3}\frac{1}{\sqrt{2\left\lvert \vec p \right\rvert}}\sum_{\lambda=0}^3(\epsilon_{\mu}^{\lambda}(\vec p)a_{\vec p}^{\lambda}e^{i\vec p \cdot \vec x}+\epsilon_{\mu}^{\lambda*}(\vec p)a_{\vec p}^{\lambda\dagger}e^{-i\vec p \cdot \vec x})\tag{2}$$

The creation/annihilation operators satisfy the following commutation relations:

$$[a_{\vec p}^{\lambda},a_{\vec q}^{\sigma\dagger}]=-\eta^{\lambda\sigma}(2\pi)^3\delta^{(3)}(\vec p - \vec q)\tag{3}$$ $$[a_{\vec p}^{\lambda},a_{\vec q}^{\sigma}]=[a_{\vec p}^{\lambda\dagger},a_{\vec q}^{\sigma\dagger}]=0$$

(The metric signature is (+,-,-,-)). Then we can compute the Hamiltonian density by considering the Legendre transform of eq. 1 (with $\zeta=1$), and then integrate over space to get the total Hamiltonian. Plugging in the expansion in eq. 2, we end up with the expression

$$H=-\frac{1}{2}\int\frac{d^3p}{(2\pi)^3}\left\lvert \vec p \right\rvert\eta_{\lambda\sigma}(a_{\vec p}^{\lambda\dagger}a_{\vec p}^{\sigma}+a_{\vec p}^{\lambda}a_{\vec p}^{\sigma\dagger})\tag{4}$$

At this point, the usual procedure would be to just normal order the Hamiltonian, to get the final expression

$$H=-\int\frac{d^3p}{(2\pi)^3}\left\lvert \vec p \right\rvert\eta_{\lambda\sigma}a_{\vec p}^{\lambda\dagger}a_{\vec p}^{\sigma}\tag{5}$$

This is the expression I've seen in several sources. However, in reality there is of course a residual c-number term in the Hamiltonian, usually identified as the vacuum energy of the quantum field. Upon normal ordering, we neglect it, but if we calculate it for this case, by applying the commutation relation eq. 3 to the second term in eq. 4, we get the term

$$H_{vacuum}=4\frac{1}{2}\int{d^3p}\left\lvert \vec p \right\rvert\delta^{(3)}(0)\tag{6}$$

Hence, there is a degeneracy of 4 for a given momentum (each mode contributing an energy $\frac{1}{2}\left\lvert \vec p \right\rvert$, corresponding to the usual $\frac{1}{2}\hbar\omega$ for each vacuum mode). In other words, all the four polarization modes, including the timelike and longitudinal ones, seem to contribute to the vacuum energy. This is in contrast to what you get when you quantize in Coulomb gauge, where of course only the two transverse polarization modes contribute to the vacuum energy. I believe this is also supposed to be the correct formula for the vacuum energy of the EM field; the timelike and longitudinal modes are not supposed to contribute, only the two transverse modes. When you quantize in Lorenz gauge, you of course have to restrict the Hilbert space to a subspace containing the "good"/physical states of the photon, and when you do this, you find that the timelike and longitudinal states go away, meaning they're unphysical, as we'd expect. However, the term in eq. 6 is a c-number (ie. it's not an operator), so projecting onto the physical Hilbert space won't change it. Hence, the longitudinal and timelike vacuum modes would still seem to contribute.

This surely can't be ignored, either. If the vacuum energy was given by eq. 6, then that would presumably have consequences or the calculation of the Casimir force; it would be twice the amount we'd usually expect (if the degeneracy really is 4, rather than 2, as is usually assumed when calculating the Casimir force). Moreover, the expression for the vacuum energy would presumably be important for cosmology, where it may contribute to dark energy. Also, I'm not sure how to interpret the fact that quantizing using the Coulomb or the Lorenz gauge seems to make different predictions here, when they're supposed to be equivalent.

So what has gone wrong here?

Answer 1

You are forgetting the contribution of the Fadeev-Popov ghosts. Although the ghosts decouple from the physical (transverse) modes in an abelian gauge theory like QED they still give a contribution to the vacuum energy that canels the vacuum energy of the unphysical longitudinal and time-like modes, and so reduces the "4" to "2".

This cancellation is especially important in calculations of the QED Casimir efect.

Answer 2

The Hamiltonian is not necessarily the vacuum energy (see e.g. this answer of mine for more on the difference between the Hamiltonian and energy), and in non-gravitational theories there is no way to measure vacuum energy anyway, and what the idealized Casimir effect cares about is a difference (or rather, the infinitesimal version, a derivative) between vacuum energies as the distance of the plates varies.

There is no reason you should treat the unphysical modes as constrained by the plates in the Casimir effect since it is the assumption of the plates being perfect conductors and hence constraining the electric field to zero what constrained the physical modes to "end" on the plates in the form of standing waves. So the unphysical modes between the plates have no contribution to the Casimir effect at all since their contribution to the vacuum energy does not change as you move the plates. (If for some strange reason you do decide that conducting plates can magically constrain the unphysical modes, then you need to do what mike stone says and also have the ghosts magically constrained in the same fashion)

Finally, the Casimir effect is only related to vacuum energies in an idealized setup with perfect conductors/infinitely strong coupling, see this answer of mine. It is therefore at least questionable to assume that inconsistencies in computations of vacuum energies have practical relevance because they "might change the Casimir effect".

Answer 3

In a sense, different gauges in EM Lagrangian can cause similar effects on the value of Hamiltonian as different coordinate systems used to formulate Lagrangian function in classical mechanics do. They change the implied Hamiltonian, even though the system is the same and its energy is the same. Consequently, analyzing different gauge Lagrangians will not provide any clue as to how to define uniquely total energy in the system. That has to come from other assumptions. In classical mechanics, we define total energy as kinetic plus potential energy where the potential energy is fixed by some non-discutable convention, e.g. for spring it is $\frac{1}{2}kx^2$, not $\frac{1}{2}kx^2 + 5J $, even though both would work in Lagrangian. We choose the former because it is the simplest and minimal. In EM theory, we usually stick to Poynting's definition, for similar reasons, unless this is invalid (such as for point particles).