Cut-off energy necessary to avoid vacuum catastrophe

Question

My understanding is that to obtain a finite vacuum energy density prediction from QFT, one must choose a cut-off point for the maximum allowed energy of a photon. Two seemingly natural choices are the Planck energy, which gives the oft-cited $10^{112}$ ergs/cubic cm figure, and the electroweak energy, which I recall reading gives a figure closer to $10^{40}$ ergs/cubic cm. My question then is: what cut-off would be required to give the value derived from cosmological observations ($10^{-8}$ ergs/cubic cm), and are photons above this cut-off known to exist?

Answer 1

The vacuum energy density of a quantum field is generically given by the sum of the ground state energies of each oscillator mode composing the quantum field - I will use the term "cosmological constant" synonymously with the total vacuum energy density. In the continuum, each mode of a free scalar field contributes an energy $\frac12\hbar\omega_k$, where $\omega_k$ is the frequency of the oscillator, and the bare (unregularised) value for the vacuum energy density is given by an integral over all the zero modes:

$$ \rho=\frac1V\langle\hat H\rangle=\hbar\int\frac{\mathrm d^3 k}{(2\pi)^3}\frac{\omega_k}2 \\=\hbar\int\frac{\mathrm d^3 k}{(2\pi)^3}\frac{\sqrt{k^2+m^2}}2 \\=\frac{\hbar}{4\pi^2}\int_0^\infty\mathrm dk\ k^2\sqrt{k^2+m^2} $$ and is naïvely UV infinite without regularisation, since modes of arbitrarily short wavelength are taken into account.

We choose to impose a hard momentum cutoff $\Lambda$ to regularise the integral. The choice of energy scale as the cutoff reflects, roughly, our confidence in that QFT is predictive at and up to this energy scale, and that above this scale, QFT breaks down, "new" physics is required to explain phenomena above this scale, and modes above the cutoff do not contribute in the QFT prediction. $$ \\\rho=\frac{\hbar}{4\pi^2}\int_0^\Lambda\mathrm dk\ k^2\sqrt{k^2+m^2} $$

Although, strictly speaking, we should perform this zero-mode analysis in terms of an interacting, photon field rather than a free scalar field, this does not change the final result appreciably, and so this calculation is accurate.

For example, this is $\rho_\gamma=\frac{\hbar\Lambda^4}{16\pi^2}$ for massless photons. More generally, the vacuum energy density goes as $\rho\sim\hbar\Lambda^4$, which you can see via dimensional analysis alone.

Anyway, if we make the claim that our theory is valid up to the Planck scale, we can set the cutoff equal to the Planck mass $\frac{1}{8\pi G}\sim10^{18}\ \mathrm{GeV}$ whereupon $\rho$ is ${\sim}10^{110}\ \mathrm{erg/cm^3}$. Importantly, there are also natural contributions from other (even composite) fields due to broken symmetry phases, notably at the electroweak breaking scale, and at the QCD scale due to chiral symmetry breaking. However, these contributions are minuscule if we integrate up to the Planck length.

This theoretical prediction is 120 orders of magnitude greater than the upper bound on the actual cosmological constant, which is on the order $10^{-10}\ \mathrm{erg/cm^3}$.

By the dimensional relation between $\rho$ and $\Lambda$, we see that 120 orders of magnitude difference between $\rho_\text{Predicted}$ and $\rho_\text{Observed}$ corresponds to a 30 order magnitude difference between $\Lambda_\text{Planck}$ and $\Lambda_\text{Match}$, which is the cutoff scale that we need to choose in our analysis to match the observed value of the cosmological constant.

So the momentum cutoff that reproduces the observed vacuum energy density is $10^{18}/10^{30}\ \mathrm{GeV} \sim 10^{-3}\ \mathrm{eV}$, or alternatively a wavelength of around $1\ \mathrm{mm}$. In the grand scheme of things, these are ridiculously low-energy photons: for instance, a photon of visible light has about 2000 times as much energy as the photons at the cutoff. In other words: yes, we have observed photons above this cutoff (and in fact you are doing so right now by reading this answer).