# Why classical probability is insufficient for quantum mechanics

I've started reading Brian Hall's Quantum Mechanics for Mathematicians. He gives a motivation for the operator formalism for quantum mechanics. If you think of position of a particle as a random variable $$X$$, and if $$\psi(x)$$ is the wave function, then $$|\psi(x)|^2$$ gives the density of $$X$$, and so $$\mathbb{E}(X)=\int_{\mathbb{R}}x|\psi(x)|^2dx.$$ At the same time, this is $$\langle\psi, \hat{X}\psi\rangle$$, where $$\hat{X}$$ is the position operator. I understand this is just motivation, but why is this necessary? It's just a way to rewrite an expectation. I know there is a non-commutative probability theory for computing probabilities in quantum mechanics, but I'd like to know why this is necessary. Why does the interpretation that position and momentum are classical random variables fail? What experiments demonstrate this?

There are four examples that come to my mind.

In the double-slit experiment, if you consider a sequence of electrons as independent, identically distributed Bernoulli random variables, the interference pattern seems to just be a consequence of the central limit theorem, so I don't see why classical probability is not suited in that example.

The uncertainty principle puts a lower bound on the product of variances of position and momentum, but would that contradict an interpretation of position and momentum as random variables?

I read somewhere that the Stern-Gerlach experiment shows that if you consider $$X$$ and $$Z$$ as random variables measuring the $$x$$ and $$z$$ spin of an electron, then $$\mathbb{E}(XZ)\not=\mathbb{E}(ZX)$$. This would be what I'm looking for, showing the random variables do not commute. However, I'm not sure I understand the argument/experiment. If this is indeed correct and someone could explain this in some detail, I would appreciate that.

Bell's inequality seems to say that if you have three random variables $$X_1,X_2,$$ and $$X_3$$ defined on a probability space taking values in $$\{0,1\}$$, then their inner products satisfy $$|\langle X_1,X_2\rangle+\langle X_2,X_3\rangle|\leq 1-\langle X_1,X_3\rangle$$ Is this not satisfied in the Stern-Gerlach experiment?

• Possible duplicates: physics.stackexchange.com/q/116595/2451 and links therein. Commented Dec 16, 2023 at 11:37
• Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on Physics Meta, or in Physics Chat. Comments continuing discussion may be removed.
– Buzz
Commented Dec 17, 2023 at 3:23

Let's say $$H$$ is a Hilbert space, $$Q, P$$ are self-adjoint operators on $$H$$ and $$\psi$$ a unit vector in $$H$$ describing a state of a system. The expectation is defined as $$\Bbb{E}_\psi(X) = \langle \psi, X\psi\rangle$$ for an operator $$X$$. Now, $$\begin{split} \Bbb{E}_\psi(QP) = \langle \psi,QP\psi\rangle = \langle Q\psi, P\psi\rangle\\ \Bbb{E}_\psi(PQ) = \langle \psi,PQ\psi\rangle = \langle P\psi, Q\psi\rangle=\overline{\langle Q\psi,P\psi\rangle} \end{split}$$ and generally, $$\langle Q\psi, P\psi\rangle\ne \overline{\langle Q\psi,P\psi\rangle}$$. In fact $$\Bbb{E}_\psi(QP) - \Bbb{E}_\psi(PQ) = \langle \psi, [Q,P]\psi\rangle$$ where $$[Q,P]$$ is the commutator which for position and momentum is $$[Q,P]=i\hbar$$ and then $$\langle \psi, [Q,P]\psi\rangle = i\hbar\langle\psi,\psi\rangle = i\hbar$$.
From the Cauchy-Schwarz inequality you get $$|\langle Q\psi, P\psi\rangle| \le \|Q\psi\|\|P\psi\| = \sqrt{\langle \psi,Q^2\psi\rangle\langle \psi,P^2\psi\rangle}=\sqrt{\Bbb{E}_\psi(Q^2)\Bbb{E}_\psi(P^2)}$$ but also $$\Im(\langle Q\psi,P\psi\rangle) = \frac{1}{2i}(\Bbb{E}_\psi(QP) - \Bbb{E}_\psi(PQ)) = \frac{1}{2i}\Bbb{E}_\psi([Q,P])$$ and since $$|\Im(\langle Q\psi,P\psi\rangle)|\le |\langle Q\psi, P\psi\rangle|$$ we get $$\frac{1}{2}|\Bbb{E}_\psi([Q,P])|\le \sqrt{\Bbb{E}_\psi(Q^2)\Bbb{E}_\psi(P^2)}$$ which is the Heisenberg uncertainty principle and specifically for $$Q, P$$ position and momentum says that $$\frac{\hbar}{2}\le \sqrt{\Bbb{E}_\psi(Q^2)\Bbb{E}_\psi(P^2)}$$
Now, let $$\mu^Q, \mu^P$$ be the projection-valued measures associated with $$Q, P$$ from the spectral theorem. If $$I_1, I_2\subseteq \Bbb{R}$$ are any intervals, then in general, when $$Q, P$$ don't commute, the projections $$\mu^Q(I_1), \mu^P(I_2)$$ don't commute either. This means $$\Bbb{E}_\psi(\mu^Q(I_1)\mu^P(I_2))\ne\Bbb{E}_\psi(\mu^P(I_2)\mu^Q(I_1))$$. But in classical probability theory, $$\Bbb{E}_\psi(\mu^Q(I_1))$$ is the probability of finding $$Q$$ in the interval $$I_1$$, $$\Bbb{E}_\psi(\mu^P(I_2))$$ is the probability of finding $$P$$ in the interval $$I_2$$ and $$\Bbb{E}_\psi(\mu^Q(I_1)\mu^P(I_2))$$ is supposed to be the joint probability of finding $$Q$$ in the interval $$I_1$$ and $$P$$ in the interval $$I_2$$. But this probability is not well-defined, because when $$Q, P$$ don't commute, $$\mu^Q(I_1)\mu^P(I_2)$$ is generally not a projection so this expectation is not necessarily positive. It can be complex. You get a different value in the opposite order $$\mu^P(I_2)\mu^Q(I_1)$$. So the joint probability distribution of two non-commuting self-adjoint operators is not defined. In classical probability, when you have two random variables on the same measure space, their joint probability is always defined.