Why is an operator the quantum mechanical analogue of an observable?

I used to think because that, if objects are treated as waves, then using operators is the necessary thing to do in order to "retrieve" the observable from a given wavefunction.

For example, in position-space and a plane wave,

$$\hat p = - i\hbar \frac{\partial}{\partial x}$$

$$\psi(p) = \frac{1}{2 \pi \hbar} e^{\frac{i}{\hbar} p x}$$

$$\implies \hat p \left(\frac{1}{2 \pi \hbar} e^{\frac{i}{\hbar} p x}\right) = p \ \psi(p)$$

But this only works for plane waves. Most wavefunctions I can think of are not eigenfunctions of the momentum operator, so I clearly have it wrong.

• Essentially because experimentally one can see that energy levels (or angular momentum or the like) are discretised and happen to be exactly spaced as eigenvalues of some matrices. This implies that more or less the theory must be a linear operator theory on a Hilbert space. – gented Apr 24 at 12:21
• But all functions we deal with can be written as a linear combination of plane waves. – FGSUZ Apr 24 at 13:40
• I feel like the title and the text don't match. The title is a kind of deep question that requires us to work backward from experiments to the mathematical structure that allows us to predict the outcomes of those experiments. The text is more about understanding the structure itself rather than understanding where it comes from. – march Apr 24 at 22:12

In mathematics, in order to describe objects that: do not commute, can be summed, can be multiplied, can be rescaled, and can be "complex conjugated" (the last mostly for convenience, admittedly); one needs to use a so-called noncommutative *-algebra.

If in addition one endows such objects with a notion of "magnitude" (mathematically a norm, physically the maximal magnitude of the possible observable values), then the natural mathematical concept is that of a noncommutative C*-algebra.

Noncommutative C*-algebras can always be concretely realized as noncommuting operators on some Hilbert space (this result is called Gelfand-Naimark Theorem).

Therefore, the latter is indeed the natural mathematical way of representing non-commutative objects that have the properties of quantum observables.

Any introductory quantum mechanics text book will cover this. The basic idea is that physical quantities that could in principle be observed, such as position, momentum, angular momentum, energy, are associated with operators. We say that each such quantity is represented by its associated operator. This is a mapping between a physical thing (e.g. momentum) and a mathematical thing (the momentum operator $$\hat{\bf p}$$).

These operators connect to physical predictions in mainly two ways. First, they may enter into the Hamiltonian operator which governs evolution over time through Schrodinger's equation. Second, each operator has a set of states associated with it---the eigenstates---and their corresponding eigenvalues. But as you have noted, the system does not have to be in one of those special states. However, whatever state $$\psi$$ the system happens to be in can always be written as a superposition of eigenstates $$u_n$$ of whatever (Hermitian) operator you might take an interest in: $$\psi({\bf x},t) = \sum_n a_n(t) \, u_n({\bf x})$$ where $$a_n$$ are complex coefficients. In this way you can determine how much momentum (or whatever) the system has at any given time. It typically won't have a single well-defined momentum (for example) but a spread of values corresponding to whichever $$a_n$$ are nonzero. A measurement of that property would give the $$n$$'th eigenvalue with probability $$|a_n|^2$$.

I hope this answer is helpful, but I must admit that I don't think you will really be able to understand this answer until you have explored quantum mechanics a bit more fully.

Let's try a somewhat more example-driven answer.

Take an electron in the state $$\psi\left(x\right)=\frac{1}{\sqrt{2\pi\hbar}}e^{i\frac{p}{\hbar}x}$$ (assume $$p>0$$). As you said, the momentum operator $$\hat{p}=-i\hbar\frac{\partial}{\partial x}$$ retrieves the momentum by a simple relation

$$\hat{p}\psi\left(x\right)=p\psi\left(x\right)$$

So this is an electron moving to the right with momentum $$p$$. All good.

Now take a look at another perfectly valid state, $$\psi\left(x\right)=\frac{1}{\sqrt{\pi\hbar}}\cos\left(\frac{p}{\hbar}x\right)$$. Applying the momentum operator this time gives

$$\hat{p}\psi\left(x\right)=-i\hbar\frac{\partial}{\partial x}\frac{1}{\sqrt{\pi\hbar}}\cos\left(\frac{p}{\hbar}x\right)=ip\frac{1}{\sqrt{\pi\hbar}}\sin\left(\frac{p}{\hbar}x\right)$$

Yikes. This is in no way $$\left({\rm some\:constant}\right)\cdot\psi\left(x\right)$$. So we can ask ourselves: what is the momentum of this state? To answer this question you need to understand that we are not reading the information correctly, or maybe even expecting something wrong. Let us rewrite our state differently

$$\psi\left(x\right)=\frac{1}{\sqrt{\pi\hbar}}\cos\left(\frac{p}{\hbar}x\right)=\frac{1}{\sqrt{2}}\color{red}{\frac{1}{\sqrt{2\pi\hbar}}e^{i\frac{p}{\hbar}x}}+\frac{1}{\sqrt{2}}\color{blue}{\frac{1}{\sqrt{2\pi\hbar}}e^{-i\frac{p}{\hbar}x}}$$

Now look what happened. This state has in fact two components - one with momentum $$\color{red}{p}$$ and one with momentum $$\color{blue}{-p}$$. It is as simple as that, our state simply doesn't have a single value of momentum, but rather two. An electron in this state moves to the right and to the left at the same time.

So $$\hat{p}$$ knows how to retrieve the momentum of its eigenstates. Want to find to momentum of a general $$\psi$$? Just write it as a sum of eigenstates of $$\hat{p}$$. Let's call them $$\psi_{p}$$, and write

$$\psi=\sum_{p}c_{p}\psi_{p}$$

Each $$\psi_{p}$$ have definite momentum $$p$$, and your $$\psi$$ has a component of fraction $$\left|c_{p}\right|^{2}$$ with momentum $$p$$.