After two recent questions on this (here's the other one), I felt it was time to write up a long account on why there are no “local non-realist” theories which can explain Bell Correlations. (A few caveats below.)
The modern structure of Bell’s Theorem is a proof by contradiction, running as follows: 1) make certain precise assumptions, 2) from these assumptions, prove an inequality (say, CHSH) that can be expressed operationally, 3) notice an experimental violation of these inequalities, 4) conclude that at least one of the initial assumptions from (1) must be false in any model which can give the right results for these experiments.
Unfortunately, the package of assumptions from (1), as expressed in Bell’s later papers and other modern proofs (not [Bell, 1964]!), has somehow been associated with the phrase “local realism”. Maybe it shouldn’t matter – it’s just words, after all – but it’s not the phrase that Bell used. Bell used “local causality” or “locality” for short. But the words “local realism” have taken on a life of their own, and confused far too many people (including some here) into thinking that the assumptions break down into “locality” and “realism”, and we can choose one of these to discard, independent of the other. Well, let’s see.
(Here I’m following the notation in https://arxiv.org/abs/1906.04313 , because it breaks out the controllable inputs into subscripts, and therefore explicitly prevents the crazy move of assigning probability distributions to Alice’s and Bob’s settings. They can choose whatever settings they want, with any distribution whatsoever, of course. We shouldn’t and can’t model Alice!)
Operational Variables:
$a$: Alice’s setting (in region 1)
$A$: Alice’s output (in region 1)
$b$: Bob’s setting (in region 2)
$B$: Bob’s output (in region 2)
$c$: Source preparation setting
Other variables:
$\lambda$: all variables associated with the spacetime region $\Lambda$ in the above figure. (These are not “operational” variables in that they can’t be directly observed.)
(Notice that $\lambda$ must be associated with this particular spacetime region, which implies that $\lambda$ is the sort of variable which can be a function over spacetime, like electric fields and particle positions. It does not include wavefunctions, since wavefunctions do not have a general spacetime representation. Multi-particle wavefunctions are functions in Hilbert or configuration space, not spacetime (x,y,z,t).)
Now, what does Bell’s Theorem attempt to restrict? Models of entanglement. Models which correctly predict the joint probability P(A,B) given any choice of inputs (a,b,c). Since inputs are special (they don’t have probability distributions), I’ll write such a model using subscripts as inputs. A model of these phenomena must, at minimum, generate a probability distribution of the outputs given any inputs, which I will write as just the probability distribution
$P_{a,b,c}(A,B)$ .
Obviously there are models which can do this correctly (say, quantum mechanics). But can they do it in a “local” way? To answer that, we need to define what we mean by locality. Bell found a nice way to do that, simply by allowing for the possibility that a deeper-level model might also make predictions of $\lambda$ along with the outcomes $(A,B)$. A model doesn’t have to do this, of course! But if it doesn’t, the model is either too sketchy/incomplete to judge its locality, or else it's complete and there’s just a big empty set for the variables $\lambda$ (maybe there’s nothing “real” in region $\Lambda$, perhaps). More on this case below.
Bell’s Theorem concerns all possible deeper-level models $P_{a,b,c}(A,B,\lambda)$. It makes assumptions about such models and then proves a contradiction with experiment.
Enter step (1) of Bell’s Theorem. The assumptions needed to prove the CHSH inequality are Bell’s Screening Assumption (BSA) and Statistical Independence (SI). I’ll take them one at a time.
Bell’s Screening Assumption (BSA): Full knowledge of $\lambda$ is just as informative about region 1 as is knowing the values in both $\lambda$ and region 2. (and vice-versa). So, BSA corresponds to the equations:
$P_{a,b,c}(A|B,\lambda) = P_{a,c}(A|\lambda) $
and vice-versa:
$P_{a,b,c}(B|A,\lambda) = P_{b,c}(B|\lambda) $
BSA is Bell’s precise definition of “locality”. If BSA isn’t true, then even if you already know all about $\lambda$, learning something about region 2 can tell you something about region 1 that you didn’t already know. Perhaps there’s been some sort of magic nonlocal or faster-than-light connection from “2” to “1”, bypassing the spacetime region $\Lambda$. Maybe this happens via a Hilbert-space wavefunctions. But if it happens, however it happens, this is “nonlocal” behavior, by definition. That’s what nonlocality means in this context: violating the above equations. (Dreaming up some other definition for locality, like “you can’t signal faster than light”, is fine, I suppose, but it’s not the “locality” ruled out by Bell’s Theorem. And it’s confusing, to have so many definitions of locality flying around, when we’re talking about this particular context.)
We’re not done yet; you can’t prove any interesting inequalities from BSA alone. You also need SI:
Statistical Independence (SI): Future settings can’t be correlated with past variables. Sometimes called “no-retrocausality”. Specifically, in this case:
$P_{a,b,c}(\lambda) = P_c(\lambda)$
(because $\lambda$ is in the past of the settings $a,b$, but not $c$.)
Okay, so Bell’s Theorem is the proof that BSA + SI => CHSH inequality, and that inequality is false. Therefore either BSA is wrong or SI is wrong (or both). That’s Bell’s Theorem.
Now, the question. Is there room for a “realism” assumption? Can you have a “local non-realist” theory which explains the entanglement correlations? Here are 4 ideas along those lines; of course I’d be happy to consider others.
Idea #1: What if “realism” is the assumption that there is something in the region $\Lambda$? Let’s define a non-realistic theory to be one where $\lambda$ is the empty set. There’s nothing there.
Implication: If $\Lambda$ is the empty set, then BSA is provably false. The right side of those equations is just the marginal probabilities measured by Alice and Bob. And the left side is clearly more informative than the right; knowing the distant setting and results gives us more information about the local results. Since BSA is violated, this is “nonlocal”. There’s no “local-nonrealistic” option here.
Idea #2: What is realism is the assumption that there is a deeper-level model $P_{a,b,c}(A,B,\lambda)$ in the first place, and a non-realistic theory should just be of the form $P_{a,b,c}(A,B)$? Well, you can define realism that way, but consider this is no different from idea #1. $\lambda$ has just disappeared. BSA fails. It’s still nonlocal.
Idea #3: Maybe “non-realism” means there are many universes with different outcomes (A,B), so in a non-realist context we shouldn’t even be talking about single outcomes.
Okay, but then you can’t even talk about a model $P_{a,b,c}(A,B)$ in the first place, $\lambda$ or no $\lambda$. So BSA is still violated! (Those equations certainly can’t be true if we’re not allowed to talk about outcome probabilities in the first place.). Bell’s “locality” is nowhere to be found in Everett/Many-Worlds. You could invent some new definition of locality, I suppose, but it woudn’t be the sort of spacetime-based locality we’re familiar with.
Idea #4: What if we define SI to be “realism”?
Well, now there’s something to talk about. It’s a terrible term to use, I think, for an assumption whose violation looks like:
$P_{a,b,c}(\lambda) \ne P_c(\lambda)$
such that the future settings are correlated with past hidden variables. That looks like (hidden) retrocausality, not “non-realism”, in my book. But, sure. If someone wants to say that violating SI but not BSA is a local non-realist theory in this sense, that’s a linguistic quirk I’d accept. But I don’t think that a single person in these threads referring to mysterious “local-nonrealist” theories is actually thinking about retrocausality. I’d be more than happy to hear otherwise.
Final Summary: Using Bell's precise definition of "locality", there are no local-nonrealist theories by any definition of realism -- except possibly “realism = no retrocausality”.
