Must observables be Hermitian only because we want real eigenvalues, or is more to that?

Question

Because (after long university absence) I recently came across field operators again in my QFT lectures (which are not necessarily Hermitian):

What problem is there with observables represented by non-Hermitian operators (by observables, I obviously don't mean the tautological meaning "Hermitian operators")?

One Problem sure is that there are not real eigenvalues. But If I say I want to "measure" some complex quantity, then that alone shouldn't be a problem, I'd be fine with complex eigenvalues then.

This question is answered stating that operators are Hermitian "if and only if it is diagonalizable in an orthonormal basis with real eigenvalues". This doesn't yet seem like a game-stopper to me, as long as I still get an orthonormal basis.

And splitting an arbitrary operator $\hat{O}$ into a Hermitian and an anti-Hermitian part, that would correspond to real and imaginary part of the observable, I could take expectation values just fine.

But maybe I'm forgetting about something, and there are other good reasons why non-Hermitian operators will lead so serious problems.

Answer 1

At the simplest level it is because Hermitian, or more precisely self adjoint, operators have a complete set of eigenstates. The existence of a complete set is essential for the propability interpretation of QM. That the eigenvalues are real is of less importance. Consider, for exmple, the operators $X,Y$ of the $x$ and $y$ coordinates of a particle. One might want, for some reason the the combination $x+iy=z$, which are the eigenvalues of $X+iY$.

To allow operators such as $X+iY$ to be observables, one can relax the self-adjoint condition to allow normal operators which are defined to be those that commute with their adjoint: $[A,A^\dagger]=0$. In this case we can decompose $$ A= \frac 12 (A+A^\dagger) +\frac 12(A-A^\dagger), $$ and as $$ [(A+A^\dagger), (A-A^\dagger)]=0 $$ we can simultaneously diagonalize the hermitian $(A+A^\dagger)$ which has real eigenvalues and $(A-A^\dagger)$ which, being skew hermitian and therefore $i$ times a hermitian operator, has purely imaginary eigenvalues. Then $A$ can be an observable with complex eigenvalues.

Answer 2

In the standard formulation of quantum theory observables are selfadjoint operators if referring to physical quantities whose values are described by real numbers. (A straightforward generalization, when admitting observables attaining complex values, is to represent them in terms of normal operators and this does not affect the discussion below.)

There are many motivations for this assumption. One, which can be traced back to von Neumann, relies upon the basic formulation in terms of elementary YES-NO propositions described in terms of orthogonal projectors, labeled by subsets (Borel sets) $E$ of $\mathbb{R}$: $P(E)$.

$P(E)$ corresponds to the proposition/elementary observable

"the outcome of the measurement of the considered observable belongs to the real subset $E$".

If $\rho$ state of the system, $tr(\rho P(E))$ is the probability that the outcome of the observable belongs to $E$.

These families of projectors are projection valued measures, PVM, and their integrals $$A = \int_{\mathbb{R}}\lambda dP_\lambda$$ are selfadjoint operators. Vice versa a self adjoint operator $A$ uniquely define a PVM $\{P^{(A)}(E)\}_{E\in B(\mathbb{R})}$ through the spectral theorem such that $$A = \int_{\mathbb{R}}\lambda dP^{(A)}_\lambda$$ The correspondence $$A\quad \leftrightarrow \quad \{P^{(A)}(E)\}_{E\in B(\mathbb{R})}$$ is one-to-one. In this sense observables are selfadjoint operators.

This approach can be found in von Neumann's textbook, in Varadarajan's textbook and in other books on the foundations of quantum theory (including a pair of mine).

More recently, a more elaborated view has been presented in terms of quantum operations. A reason of this investigation is the attempt to define a realistic notion of the post measurement state $\rho'$. The standard notions based on von Neumann's, Luders' and von Neumann-Luders' postulates $$\rho' = \frac{P^{(A)}(E)\rho P^{(A)}(E)}{tr(\rho P^{(A)}(E))}\:,$$ is nowadays considered quite unrealistic, also in view of a more elaborated quantum technology at our disposal.

Within this new approach the YES-NO elementary observables are replaced by so-called POVMs: measures valuated in terms of positive operators $\{Q(E)\}_{E\in B(\mathbb{R})}$ with $0\leq Q(E) \leq I$.

The physical genesis of this notion is a bit involved and based on an indirect measurement procedure which does not destroy the measured system. As a matter of fact, one measures, with a standard procedure, a second system S' (destroying it) that had a given (known) interaction with initial system S we want to measure. If $\rho$ is the (generally mixed) state of $S$, $tr(\rho Q(E))$ is the probability that the outcome of the observable belongs to $E$. The net effect on S is described, as said, with a POVM instead of a PVM. The advantage of this procedure is that it permits to control also the post measurement state of the system S.

A quick account on these ideas can be found in a nice paper by P. Busch [1].

Coming back to the main issue, even referring to POVMs, Hermitian operators pop out in any cases as consequence of remarkable results by Naimark.

If $\{Q(E)\}_{E \in B(\mathbb{R})}$ is a (normalized) POVM over the real line $\mathbb{R}$, there is a Hermitian operator $A$ associated to it, the unique satisfying $$\langle \psi| A \phi\rangle = \int_{\mathbb{R}} \lambda \mu_{\psi,\phi}^{(Q)}(\lambda)$$ where (with some details I omit here on the domain of $A$) $$\mu_{\psi,\phi}^{(Q)}(E) = \langle \psi| Q(E) \phi\rangle\:.$$ Vice versa (and this is the aforementioned result by Naimark) every Hermitian operator $A$ with dense domain $D(A)$ can be decomposed as above by a POVM (with some details on the domain). That POVM is unique only if $A$ is maximally symmetric and is a PVM only if $A$ is selfdajoint.

These results can be found spread in the literature. A good (but very wide) reference is the beautiful book by Busch and collaborators [2]. A summary of the interplay of POVMs and Hermitian operators can be found in a recent paper of mine and N. Drago [3]

TECHNICAL NOTE If $A: D(A) \to H$ is a linear operator in the Hilbert space $H$, $D(A) \subset H$ being a subspace,

(i) $A$ is Hermitian if $\langle x| Ay\rangle = \langle Ax| y\rangle$ for every $x,y \in D(A)$;

(ii) $A$ is symmetric if it is Hermitian and $D(A)$ is dense.

(iii) $A$ is selfadjoint if it is symmetric and more strongly $A=A^\dagger$, where $A^\dagger : D(A^\dagger) \to H$ is the adjoint operator.

---

[1] P. Busch, No Information Without Disturbance: Quantum Limitations of Measurement in Quantum Reality, Relativistic Causality, and Closing the Epistemic Circle: An Interna- tional Conference in Honour of Abner Shimony, Perimeter Institute, Waterloo, Ontario, Canada, July 18-21, 2006, Eds J. Christian, W.Myrvold, Springer-Verlag, 2008, ISSN: 978- 1-4020-9106

[2] P.Busch, P.Lahiti, J.-P.Pellonpää, K.Ylinen, Quantum Measurement. Springer (2016)

[3] N. Drago and V.Moretti, The notion of observable and the moment problem for ∗- algebras and their GNS representations. Lett. Math. Phys, 110(7), 1711-1758 (2020)

Answer 3

Using real numbers to label measurement outcomes is convenient, because we can use those labels to define summary-statistics like averages and standard deviations. But using real numbers to label measurement outcomes is not necessary, neither in the real world nor in quantum theory. It may be overwhelmingly convenient in practice, but it's not necessary in principle.

Dual-purpose observables

Some of the operators that represent observables, like the angular momentum operators, serve a dual purpose. In addition to representing something that could be measured, the angular momentum operators are also the generators of rotations — which is why they're interesting as observables. For their role as the generators of rotations, their self-adjointness and their spectrum are both essential, so it's natural to use those operators as-is to represent the observable, too.

Another important example of a dual-purpose observable is the total-energy observable, called the Hamiltonian $H$. The Hamiltonian generates time-translations, and for that purpose it must be self-adjoint, because time-evolution must be unitary.

Observables like total energy and total angular momentum are prominent in textbooks becasue of their dual role as the generators of symmetries, but most observables of practical interest are local observables — a concept that gets lost in overly simple models like nonrelativistic single-particle quantum mechanics. Local observables remain important even in models that don't have any symmetries, because they represent things that can be measured within finite regions of space(time). These are the observables I mostly have in mind in the following paragraphs.

The double-commutant representation

In principle, an observable can be represented by the set of possible measurement outcomes, without any numeric labels at all. According to the most basic principles of quantum theory, each of those outcomes is represented by a projection operator. (Generalizations like using POVMs can be convenient, but we can always accomplish the same goal by using ordinary projection operators in a more comprehensive model.) The most basic way to represent of an observable is to use a set of mutually commuting projection operators, together with all projection operators that can be formed from these using sums and products and limits. The projection operators are all we need to calculate probabilities and to use the projection rule. (The projection rule is used to calculate probabilities for chronological sequences of measurement outcomes, and we also use it implicitly every time we choose an initial state based on our knowledge of how the physical system was prepared.)

More precisely, we can represent an individual observable as the set of all projection operators in a commutative von Neumann algebra. That might sound intimidating, but it's actually quite easy to define algebraically, thanks to something called the double commutant theorem. Let $\Omega$ be any collection of mutually commuting operators, together with their adjoints. The commutant of $\Omega$, denoted $\Omega'$, is the set of all operators that commute with everything in $\Omega$. Clearly $\Omega\subset\Omega'$. The double commutant of $\Omega$, denoted $\Omega''$, is the set of all operators that commute with everything in $\Omega'$. The double commutant $\Omega''$ is the smallest von Neuman algebra (self-contained with repsect to sums, products, and limits) that contains everything in $\Omega$. We can use the set of all projection operators in $\Omega''$ to represent an observable.

Consider the case $\Omega=\{A\}$, where $A$ is a single self-adjoint observable. The projection operators in $\{A\}''$ include all of the projection operators involved in the projection-valued measure (PVM) associated with the spectral decomposition of $A$. If the spectrum of $A$ is discrete, then $\{A\}''$ includes all of the projection operators onto its eigenspaces. The set of projection operators in $\{A\}''$ is the philosophically "pure" way to represent the observable $A$. We may have $\{A\}'' = \{B\}''$ even if $A\neq B$, which means that the operators $A$ and $B$ reallly represent the same observable but with different numeric labels, because $\{A\}''$ and $\{B\}''$ both have the same projection operators — they both have the same set of possible measurement outcomes, even though the operators $A$ and $B$ use different ways of labeling those outcomes.

Since the question mentioned the idea of complex eigenvalues, consider the observable $\{H\}''$, where $H$ is the Hamiltonian. The Hamiltonian is typically an unbounded operator, which means that it's not quite defined on the whole Hilbert space. That's okay, because the operators that implement time-translations are the unitary operators $e^{-iHt}$, which are defined on the whole Hilbert space, even though $H$ is not. Their "eigenvalues" (values of the spectrum) are complex, but we have the identity $$ \{H\}'' = \{ e^{-iHt}\,|\,t\in\mathbb{R}\}'', $$ which says that the set of projection operators generated by $H$ is the same as the set of projection operators generated by all of the unitary operators $e^{-iHt}$. As far as their role as observables is concerned, using $H$ is equivalent to using the whole collection of unitary operators $e^{-iHt}$, because they both generate the same set of projection operators — the same set of possible measurement outcomes.

Convenience is important, too

Nature doesn't care how we label the outcomes of a measurement, but convenience is still an important consideration. In practice, representing an observable as a single operator $H$ is usually more convenient than using the whole set of operators $\{H\}''$ or $\{e^{-iHt}|t\in\mathbb{R}\}$. The real-valued numeric labels are also convenient, for the reasons mentioned above among others. So we shouldn't be too critical of textbooks for elevating the single-hermitian-operator representation to the status of a postulate. That representation is both sufficient and convenient. Still, knowing that it's not necessary is fundamentally satisfying, even if we never exploit this freedom in practice.

Answer 4

I'd like to avoid a semantic circularity first of all, because some literature define "observable" as a Hermitean operator, or self-adjoin operator, or similar mathematical objects.

So let me ask instead two questions with a more practical or experimental point of view:

• Must the outcomes – values and statistics – of a measurement on a quantum system be described by a Hermitean operator?
• Can a measurement whose outcomes are complex numbers, or vectors, or labels such as "yes" "no", be described by a Hermitean operator?

The answer to both is no: in the most general case we use something called a "positive-operator-valued measure", of which Hermitean operators are special cases.

The modern theory of quantum measurement is much more general than the one based on Hermitean operators. A measurement is most generally represented not by a Hermitian operator, but by a so-called positive-operator-valued measure (POVM, or POV), also called a "resolution of identity" (and several other names). Mathematically it is a set of positive-definite operators that sum up to the identity operator.

POVMs allow for more general situations, such as these:

• Measurements affected by apparatus noise or experimental imperfections.
• Measurements with more outcomes than the dimension of the Hilbert space (finite-dimensional case).
• Measurements with any kind of outcomes. That is, not only real numbers, but for example also complex numbers, or tuples of numbers, or elements of a vector space, or categorical outcomes not represented by numbers: "up" and "down", "blue" "red" "green", or whatnot.

I think the last generalization is the one that interests you.

A POVM manages these generalizations because it separates the (discrete or continuous) topology of the outcome space from the specific values (numeric or not) that we can associate to the outcomes. The POVM, when combined with the quantum state (density operator) encodes the probability of each outcome without caring what the value or other label of that outcome is.

If the outcomes are numeric and we are interested in statistics such as mean and standard deviation, we can simply obtain the latter by combining the probabilities given by the POVM with the numeric values of the outcomes.

Hermitean operators are a special case of POVM: they combine together the operators that yield the probabilities of the outcomes (the eigenprojectors in this case), with the values of the outcomes in the specific case where these values are real numbers (the eigenvalues).

Mathematically the theory of POVMs can still be based on Hermitean operators, by using an auxiliary system. I personally see this as a purely mathematical possibility with no cogent physical foundation. In the end me must always choose something as a "black box" in terms of which we define the rest, and I prefer to use the POVM maths as the starting point.

Let me add that we're here speaking only about the probabilities and statistics of the outcomes of a measurement, not about the state (if any) that is produced after the measurement. Today we have a more general view of that too, which is connected to POVMs. See the texts below for this topic.

Good texts to consult about POVMs, their motivation, experimental applications (which are many), special cases, and so on, are for example:

• Bengtsson, Zyczkowski: Geometry of Quantum States: An Introduction to Quantum Entanglement, §10.1.
• Peres: Quantum Theory: Concepts and Methods, p. 282.
• de Muynck: Foundations of Quantum Mechanics, an Empiricist Approach, §3.3.
• Holevo: Probabilistic and Statistical Aspects of Quantum Theory, chap. 1.

Answer 5

The other answers are a bit above my pay grade, but I've always heard this story:

A physical transformation $U : \mathcal{H} \to \mathcal{H}$ of the Hilbert space of states $\mathcal{H}$ has to be unitary so that normalization of states is preserved. Suppose, moreover, this transformation depends smoothly on parameters $x_\alpha$ with $U(0) = 1$. Then $$ U(x_\alpha) = 1 + K_\alpha x_\alpha + O(|x_\alpha|^2) $$ for some operators $K_\alpha$. Applying the condition for unitarity $$ 1 = U(x_\alpha)^\dagger U(x_\alpha) = 1 + (K_\alpha + K_\alpha^\dagger) x_\alpha + O(|x_\alpha|^2) $$ So that $$ K_\alpha + K_\alpha^\dagger = 0 $$ Thus we see that the generators $K_\alpha$ of the transformation $U(x_\alpha)$ are anti-Hermitian. In the classical theory, observables are precisely the generators of transformations of the state space. This gives us the idea that quantum observables should have something to do with Hermiticity, since generators are anti-Hermitian. It is a (mysterious) fact that in the quantum theory observables are obtained from generators through multiplication by $-i$ (I found some comments on this fact here). Thus $$ P_\alpha = -iK_\alpha $$ are the observables corresponding to the transformation $U(x_\alpha)$. One checks $$ P_\alpha^\dagger = i K_\alpha^\dagger = -iK_\alpha = P_\alpha $$ Hence observables are (or should be expected to be) Hermitian.

---

Running the argument in reverse may be more compelling. In the classical theory, every observable (function on phase space) locally generates a physical flow of states. Suppose we want this to be true in the quantum theory. The only candidates for observables are linear functions on the Hilbert space (I'm not actually sure why they must be linear). If we want these observables to generate a physical flow on states, they must be (up to multiplication by $i$) Hermitian. If the observables are not Hermitian, the flow they generate will not preserve the normalization of states.

Answer 6

Given a separable Hilbert space that underlies a quantum system, the observables on this system are the self-adjoint operators on the Hilbert space. The reason they must be self-adjoint is that the spectrum of an operator is real if, and only if the operator is self-adjoint. The spectrum of an operator is the set of possible measurement values that you can hope to measure in an experiment. The eigenvalues of an operator are a subset of its spectrum.