Why do we need to embed particles into fields?

Question

In QFT we have the so-called embeding of particles into fields. This is discussed at full generality in Weinberg's book, chapter 5. In summary what one does is:

1. From Wigner's classification, for each $(m,j)$ with $m\in [0,\infty)$ and positive half-integer $j$ we have a unitary representation of the Poincare group characterizing one elementary particle, from which we can build a Fock space. This gives annihilation/creation operators $a(\mathbf{p},\sigma),a^\dagger(\mathbf{p},\sigma)$.

2. From these, we define $$\psi_\ell^{-}(x)=\sum_{\sigma}\int v_\ell(x;\mathbf{p},\sigma)a^\dagger(\mathbf{p},\sigma)d^3\mathbf{p},\quad \psi_\ell^+(x)=\sum_\sigma \int u_\ell(x;\mathbf{p},\sigma)a(\mathbf{p},\sigma)d^3\mathbf{p}$$

   and then we demand that there be a representation $D$ of the Lorentz group, so that

   $$U_0(\Lambda,a)\psi_\ell^\pm(x)U_0^{-1}(\Lambda,a)=\sum_{l'}D_{\ell\bar{\ell}}(\Lambda^{-1})\psi_{\bar\ell}^{\pm}(\Lambda x+a).$$

3. If $D^{(j)}$ is the Little group representation associated to $(m,j)$ the above requirement links $D$ and $D^{(j)}$.

Now, I would really like to understand why one does that, but really understand, in the sense that I actually see why this is necessary, because right now I look at this and think "ok, but why would anyone do this anyway"?

This has been hinted at @ACuriousMind answer here and I want to expand this discussion here. It is said in the answer:

The definition of the field taking values in a vector space restricts it to transform in a finite-dimensional representation, hence it cannot be one of Wigner's particles. It is important that, while fields contain the creation and annihilation operators for the particles in their mode expansion, they themselves do not transform like particles. It is the Hilbert space of a QFT that must carry the proper unitary representations, not the fields.

We need a field because it encodes the dynamics of the theory - a QFT needs a map between in and out states, given by the S-matrix, which is obtained from the field action via the path integral (or the LSZ formalism or whatever approach you are most comfortable with). The mere knowledge of the Fock spaces (via Wigner's classification) does not suffice for this.

So in some sense it seems that this whole thing of embedding particles into fields is necessary for the dynamics. This is also suggested by Weinberg, but I can't understand just like this.

Also, if we assume Weinberg's point of view, that particles come first with Wigner's theorem being truly the starting point and fields coming later, this makes even less sense.

My question: intuitively why do we need this embeding of particles into fields? Why this is necessary? How can we look at this and actually see this is necessary and understand this.

It seems to be the fundamental link between quantum fields and relativistic particles, and I still can't get the fundamental idea behind it, and that is what I want to understand.

Answer 1

The most satisfying answer I have seen to this question comes from the perspective of Algebraic Quantum Field Theory (AQFT), which is a formulation of QFT based entirely on observables, with no mention of fields at all. The pattern of observables is required to satisfy a few relatively simple conditions that I'll preview below. After the conditions are expressed in mathematically rigorous terms (which I won't do below), we can do two things: (1) we can deduce that observables can be constructed in terms of fields, and (2) we can operationally characterize the phenomena that we call particles. The central point here is that (1) and (2) are logically independent of each other. So it's not that we need to embed particles into fields; it's just that we can — and this is so incomparably convenient that we practically always do.

(This doesn't answer the historical question of how people came up with the idea of using fields, but it does address the logical question of why we might do that in hindsight.)

AQFT is based on the idea that observables should be associated with regions of space-time rather than with objects (like particles). This is the idea in other formulations of QFT, too, but whereas other formulations are concerned with convenient ways of constructing observables (which is important, of course), AQFT is concerned with what can be deduced just from the pattern of observables without making assumptions about how we might actually construct them. To give the flavor of the subject, I'll list a few of the general principles here. These principles are for QFT in Minkowski space-time.

Let $A$ denote the algebra generated by all observables, using the Heisenberg picture. In any specific model, we have a collection of subalgebras $A(O)\subset A$ associated with open subsets $O$ of Minkowski space-time. Local observables are elements of these subalgebras.

One of the general principles (microcausality or Einstein causality) says that if $O_1$ and $O_2$ are two spacelike-seaprated regions, meaning that no timelike world-line passes through both of them, then everything in $A(O_1)$ commutes with everything in $A(O_2)$. In particular, it says that spacelike-separated local observables commute with each other. This is familiar from other formulations of QFT.

Another general principle (sometimes called local primitive causality, a refinement of the time-slice axiom) says that if $O_1$ and $O_2$ are two regions such that every timelike world-line through $O_1$ also passes through $O_2$, then $A(O_1)\subseteq A(O_2)$. In particular, this says that observables in $A(O_1)$ are also members of the algebra $A(O_2)$, even if the region $O_2$ is far in the future (or past) of $O_1$. This is also familiar from other formulations of QFT in the Heisenberg picture.

AQFT also assumes the spectrum condition, which says that the spectrum of the generator of time-translations (which is an observable but not a local observable) must have a lower bound. This, again, is familiar.

The point of restating these familiar things is to emphasize that they are all expressed in terms of observables, with no mention of fields. This is important because if we wanted to define some kind of "isomorphism" in the "category" of QFTs, these observables-only structures are the ones we would want isomorphism to preserve.

More detail about AQFT can be found in http://arxiv.org/abs/math-ph/0602036, and the classic source is Haag's book Local Quantum Physics. Although texts about AQFT tend to emphasize mathematical rigor, the important point here is that it provides a good conceptual foundation.

Answer 2

As Landau and Lifshitz explain in their quantum mechanics book (page 241), the second quantization formalism is intended to allow us describe states with variable or indefinite particle numbers. Put differently, in spite of the fact that this formalism is initially based on the single particle Hilbert space of solutions, it allows us to describe states which are more general than a countable number of particles.

The most famous example where this formalism becomes necessary is the relativistic state of (indefinite number of) soft photons near charged particles.

In addition, the $\psi-$ operators get life of their own when interaction is switched on. They describe quantum fields even if they cannot be decomposed into creation and annihilation operators.

One type of states which the second quantized formalism allows us to describe is the infinite volume, finite density states, where the number of particles is infinite. There are physical phenomena which can occur only under these conditions, for example the axial anomaly. It require the existence of (an infinite number state of) a Dirac sea from which the axial charge is drawn.