What is the relation between non-contextuality and Bell's hidden variable model?

Question

While reading about contextuality in quantum mechanics, I stumbled upon the following statement (in Peres (2002), top of p. 190): in a two-dimensional Hilbert space, it is possible to construct hidden variable models (HVMs) that reproduce all the results of quantum theory. I'm trying to understand better what is meant by this.

Peres, in support to the above statement, redirects the reader to p. 159, where "Bell's model of hidden variables" is described. This works as follows:

$\newcommand{\bs}[1]{\boldsymbol{#1}}\newcommand{\ket}[1]{|#1\rangle}$We want to describe the possible results of measuring a two-dimensional state $\psi\equiv\ket\psi$. We note that any observable $A$ can be written as $A=\bs n_A\cdot\bs\sigma$ with $\bs n_A\in\mathbb R^3$ and $\sigma_i$ the three Pauli matrices. The possible outcomes corresponding to such observables, in QM, would be $\pm n_A$, where $n_A\equiv |\bs n_A|$. Let us also define $C\equiv \psi^\dagger A\psi/n_A$. We can then predict the experimental outcomes of measuring $A$, by leveraging an auxiliary hidden variable $\lambda$, as follows:

• If $-1<\lambda < -C$, then the outcome is $-n_A$;
• If $-C< \lambda < 1$, then the outcome is $+n_A$.

We recover the predictions of QM when $\lambda$ is distributed uniformly in $[-1,1]$.

As Peres points out (in p.190), this model correctly predicts the results of measuring any given observable $A$. He then goes on (in p.190) to describe Mermin's argument for the non-contextuality of QM, which relies on finding a set of two-qubit observables which cannot all be assigned a definite numerical value. He remarks that Mermin's argument requires the use of a four-dimensional Hilbert space, whereas we know that in the two-dimensional case we can build a HVM to reproduce all results of quantum mechanics, as per the argument above. This is where I get confused: isn't this comparing apples and oranges?

As I understand it, Mermin's argument is about comparing measurement results obtained in different bases. But we don't do anything of the sort when discussing Bell's HVM. In fact, can't we extend Bell's argument to arbitrary dimensions? For any given observable $A$ with $n$ distinct eigenvalues $\lambda_j$, use a uniformly distributed hidden variable $\lambda\in[-\|A\|,\|A\|]$, and say that the experimental outcome is the $j$-th whenever $\lambda\in[\lambda_j,\lambda_{j+1}]$. How is this then consistent with Mermin's argument about the non-contextuality in four dimensions?

I'm pretty sure I'm missing the point of contextuality here, and what Mermin's argument is supposed to tell us, so I'll appreciate any clarification on the matter.

Answer 1

can't we extend Bell's argument to arbitrary dimensions?

Yes, Bell's argument can be extended to contrive a hidden variables model for a Hilbert space of arbitrarily many dimensions. Contriving a contextual hidden variables model is easy!

... but it's also uninteresting. We can always contrive a theory that reproduces all experimental results perfectly, simply by taking every known result to be one of the theory's axioms. Contextual HV models aren't quite that ridiculous, but they're still in the "uninteresting" category.

In a Hilbert space with four dimensions, we can have a non-trivial observable $A$ (not proportional to the identity) that commutes with both $B$ and $C$ even though $B$ and $C$ don't commute with each other. In Mermin's words (https://arxiv.org/abs/1802.10119):

This tacit assumption that a hidden-variables theory has to assign to an observable $A$ the same value whether $A$ is measured as part of the mutually commuting set $A$, $B$, $C$, . . . or a second mutually commuting set $A,L,M,...$ even when some of the $L,M,...$ fail to commute with some of the $B, C, . . .$, is called "non-contextuality" by the philosophers.

We can extend Bell's argument to contrive a hidden variables model in a four-dimensional Hilbert space, but not one that respects this constraint. Non-contextual hidden variables models cannot reproduce quantum theory's predictions, and that's interesting.

Answer 2

I am not sure you are missing the point. We can produce a Hidden variables model to reproduce the results of quantum mechanics for a two dimensional Hilbert space, but the Kochen-Specker theorem shows that this cannot be done for a Hilbert space of dimension more than 2 (von Neumann considered infinite dimensional Hilbert space). Gleason's theorem can also be interpreted as meaning that the only probability measure on a Hilbert space of dimension at least 3 is that given by quantum mechanics, thereby excluding hidden variables).

I regard contextuality as a red herring in these arguments. It is also a red herring to consider Hilbert space of dimension 2, since that does not describe quantum mechanics. Whenever contextuality is introduced, it seems to be to confuse the issue by changing the subject (the current Wikipedia article on Gleason's theorem is a case in point; compare with the statement of Gleason's theorem which was quoted in this SE answer)

Contextuality means that the hidden variable determines just a particular measurement or class of measurements. It does not satisfy classical determinism, which means the determination of all possible measurement results, not just the result of the measurement performed. It is not sufficient for contextuality to say that the hidden variable contains parts that determine either position or momentum and that since only one of these measurements can be performed the other exists but is not used; conjugacy between position and momentum states that when one measurement is performed the hidden variable for the other measurement cannot exist.

It makes no difference that we can in practice perform only one measurement, because quantum logic describes not only the measurement performed but also what would happen in measurements which are not performed. Contextuality would require that different hidden variables pop in and out of existence whenever an experimenter changes his mind about which measurement he is to perform. This would imply different underlying physics depending on what a physicist will choose to do.

In short, whenever I see arguments about contextuality, I find physicists clinging to determinism and seeking a loophole which does not exist.