I am reading the QFT book by Peskin and Schroeder. They compute the Hamiltonian for a free quantum scalar field and find \begin{equation} H = \int \frac{d^3p}{(2\pi)^3}\omega_p (a_p^\dagger a_p + \frac{1}{2}\delta(0)). \end{equation} Then they say that the second term cannot be detected experimentally, so it will be ignored. The same refers to spatial momenta \begin{equation} P^i = \int \frac{d^3p}{(2\pi)^3}p^i (a_p^\dagger a_p + \frac{1}{2}\delta(0)). \end{equation} The second term in the formula above in the book by P-S is ignored straightaway.
I wonder to what extent it is accurate. It seems to me, that for many things to work properly, the singular term just should not be there. For example, one normally expects that single particle states $|p\rangle$ should realize a unitary irreducible representation of the Poincare group. To this end, one needs that \begin{equation} H |p \rangle = \omega_p |p\rangle, \qquad P^i |p\rangle = p^i |p\rangle. \end{equation} These formulas are spoiled if the vacuum energy is added.
Besides that, in some standard computations one actually uses that the vacuum has the vanishing energy. For example, one often uses that \begin{equation} e^{iHt}|0\rangle = |0\rangle \end{equation} when computes transition amplitudes. This is equivalent to $H|0\rangle = 0$ or, equivalently, that the vacuum is invariant under translations in time. The same one usually says about spatial translations.
In other words, it seems to me that the singular term is not just immaterial and one can do whatever one wants with it. Instead, it should actually be absent. Is that right? If it is zero, is there any clear way to explain how to get rid of it? For example, one may postulate that before replacing fields with operators, $a$ and $a^*$ should be properly ordered. After all, ordering ambiguity is, indeed, there and it should be fixed somehow (By the way, is there any systematic discussion of what is the right way to fix the ordering ambiguity before one promotes fields to operators when quantizing things? Or one just picks the most natural ordering and hopes for the best?). If it is non-zero, does that mean that the vacuum is not Poincare invariant? Is that ok?
At the same time, Wikipedia says that the vacuum energy not only does not have to be zero, but it is also measurable. For example, in the Casimir effect. This seems to contradict to the statement by Peskin and Schroeder.
Yet another thing is that people seem to compute the vacuum energy as ${\rm log}\; {\rm det} (\Box + m^2)$ . How can one see that it does compute the vacuum energy? Actually, it looks a bit surprising to me that it does, as $H$ is the time component of a vector, while ${\rm log}\; {\rm det} (\Box + m^2)$ is Lorentz invariant. And finally, how is it related to the vacuum one-loop diagram?
I would appreciate if you help clarifying these matters or give a reference where it is written in the accessible manner.
