Why are expectation values of an observable important in QM?

Question

I've been reading that expectation values of an observable is all what we can get and are the key quantities of the theory, but performing the same experiment many times would generate a distribution probability for the possible values of the observable, which is better than only expectation values. So why do we claim that 'all we can get is expectation values'? Also, why cannot we model these uncertainties in the a priori knowledge of a measurement using random variables and probability language? Is it really necessary to go through this whole formalism?

Answer 1

This is an important point which should be discussed.

What one obtains from experiments are frequencies of outcomes of given measured observables on an ensemble of identical quantum systems all prepared in a common given quantum state.

These values correspond to the theoretical probabilities of outcomes of those observables predicted by QM.

In principle, expectation values are just aggregations of those data (experimental or theoretical).

Actually, from the theoretical perspective, every observable $A$ can be decomposed, through the spectral decomposition, into elementary observables: orthogonal projectors $P^{(A)}_\Delta$, where $\Delta$ is the corresponding outcome (typically a set of values or a single value if the spectrum of $A$ is discrete). These orthogonal projectors are observables as well.

Here is the point: the expectation values of these elementary observables: $$tr(\rho P^{(A)}_\Delta)\:,$$ referred to a mixed state $\rho$, or to a pure state $\psi$ $$\langle \psi| P^{(A)}_\Delta \psi \rangle = ||P^{(A)}_\Delta \psi||^2$$ are the probability that, if the state is $\rho$, respectively $\psi$, then the observable $A$ takes value in $\Delta$.

This datum can be directly compared with the experimental frequencies of occurrence of $\Delta$ when measuring $A$.

In summary, expectation values of all observables, including the elementary ones represented by the orthogonal projectors, include the whole physical information predicted by QM which can be compared withh the experimental data.

From the theoretical side, the celeberated Gleason theorem establishes in particular that the information encompassed by the expectation values of these elementary observables uniquely permits to reconstruct the quantum state.

There are several extensions of this perspective.

(1) Sticking to the Hilbert space formulation, we can extend the plethora of elementary observables by adding the so called effects, positive and bounded by $I$ operators. They generalize the notion of orthogonal projectors and can be combined in generalized version of spectral decompositions defining the so called POVMs (Positive operator valued measures), which are at the basis of the modern formulation of the quantum measurement theory. The corresponding of the Gleason theorem is known here as the Busch theorem (whose proof is definitely easier than the one of Gleason's theorem however)

(2) On the other hand, we may extend the theory towards the algebraic approach getting rid of the structure of Hilbert space as fundamental notion. Here, the observables are formally selfadjoint elements of unital $*$-algebras or unital $C^*$ algebras.

In both cases, the physical information is still embodied in "expectation values" of elementary observables. Alternatively, especially in the case (2) the probability distribution is inferred from the expectation values of standard observables through the solution of the moment problem.

The case of $*$-algebras is actually affected by some issues, quite overlooked in the literature in this paradigm. These problems affect in particular the idea that the series of moments of an observable (the expectation values of the powers of that observable) can be used to uniquely find the probability distribution. It is certainly true if the observable is bounded. But it is generally false if it is not, how it happens in the generic case in QM.

---

The last part of the question seems to concern the possibility of building up a classical description of the quantum phenomenology, by means of classical notions of probability theory in particular. This is a deep issue with many facets and results none completely definite yet. There are several known severe obstructions against the use of naive classical like models based on "hidden variables" to account for the apparent randomness (Bell theorem, Kochen-Specker theorem, etc.)

This old answer of mine may be of some help: It should be evident that QM in Hilbert space is a probability theory. However, it is stated on a generalized notion of space of events, where quantumness shows up (the space is not Boolean, though it is a lattice).

Answer 2

The reason that people care about expectation values -- that is, the average result observed in an experiment -- is to some extent historical. In early quantum mechanical experiments, one would measure many quantum mechanical particles together. Thus, one would see their averaged signal. (Even then, however, one might be able to see more than just the average: For instance, a bunch of atoms would still radiate with different spectral lines, which one could obverse.)

Modern quantum mechanical experiments allow us to perform experiments with single quantum mechanical particles, such as atoms. In that case, it makes much more sense to describe the single quantum mechanical system and use it to predict the probability for different measurement outcomes, rather than to talk about their average. Indeed, e.g. in quantum information and computing, one rarely talks about expectation values, and much rather about the probability distribution of the outcomes.

Still, as other answers discuss, these two perspectives are equivalent in terms of what they allow to predict and infer. It is just that one or the other might be the more natural question to ask, depending on the setup one is interested in describing.

Answer 3

As an add-on to the fantastic answer of Valter Moretti I would like to say that people are in the business of working out full probability distributions for measurements (usually by the name "full counting statistics"). However, even when doing this it is usual to represent the distribution in terms of expectation values, just of more complicated operators. In particular one can consider $$ P_O = \langle \delta(\hat{O}-O) \rangle $$ which is the probability that you measure the eigenvalue $O$ of an operator $\hat{O}$. Or, if you don't like delta functions you can consider $$ \chi_\lambda = \langle \exp(i\lambda \hat{O})\rangle $$ which is the characteristic function (i.e. Fourier transform) of the probability distribution you are after. In either case though, knowing all expectation values suffices to answer questions about probability distributions. And of course, if one only cares about the variance, then it is enough to simply know $\langle \hat{O}^2 \rangle$.

Answer 4

I've been reading that expectation values of an observable is all what we can get

What you have been reading is oversimplified. By making multiple measurements on a system in an identical state (or a state that is as near identical as we can make it) we can find the eigenstates of the system and their relative probabilities. So when we run the double slit experiment, for example, we don't just know where the centre of the interference pattern is - we also know the spacing and relative brightness of the peaks in the interference pattern either side of the centre.

Answer 5

This situations is not specific to Quantum Mechanics and the reason is quite simple. Basically, whenever you need to deal with a random event experimentally (we sometimes forget that Physics is not only made of theory...), you will end up doing statistics in order to reconstruct the moments of the probability distribution behind this random event, because they are the most natural/direct/simple quantities to describe a random variable starting from the observational data.

Next, you can make good use of the probability theory in order to translate these moments into your favorite tools such as density, cumulative or characteristic functions, etc., which are all equivalent and valid ways to represent a random variable.

Answer 6

Suppose we are measuring some property $A$ of a quantum system, and the system is in a superposition of states with different values of $A$:

$$ \Psi = \sum c_i \phi_{Ai} $$

Before we do the measurement we cannot predict what the result of the measurement will be. The best we can do is say that the probability of getting the result $A_i$ is $c_i{}^2$, and that the average result if we do many measurements will be the expectation value $\langle\Psi|A|\Psi\rangle$.

You don't cite a source for the statement you quote:

expectation values of an observable is all what we can get and are the key quantities of the theory

but I would imagine this is what it means. It doesn't mean that the individual values $A_i$ returned by the measurement are unimportant, but just that they are fundamentally unpredictable. By contrast the expectation value is predictable.