Why do people rule out local hidden variables?

Question

I bet the automatic response to my question would be "Bell's theorem" and of course I am not disputing Bell's proof. I am however uncertain of one of his assumptions.

The so called "no conspiracy" assumption states that we somehow posses this magical thing called free will which makes us able to break free from the causal chain of events that make us measure the $y$-axis instead of the $x$-axis and so on in experiments. Obviously this makes little sense once reviewed under the light of logic. What this naturally implies is that, if a theory is fully deterministic, it is instantly super-deterministic.

Simon Kochen and John Conway published a theorem in 2009, "The Strong Free Will Theorem", which simply-put states: either everything is deterministic (super-deterministic) or every particle has free will.

Among the few who accept this to the fullest extent is Gerard 't Hooft. He has proposed that what we call sub-atomic particles are really just templates and that there is a realm beneath the quantum where the true fabric of reality is "hidden". By accepting determinism he escapes Bell's theorem (due to not having to accept the "no conspiracy" assumption) and can construct a local, deterministic and realist hypothesis of reality. In Gerard 't Hooft's model there is cellular automata somewhere near the Planck scale which gives rise to the "emergent quantum mechanics". This means that there exist no superpositions in objective reality, the cat is always either dead or alive, no collapse of wave functions, no branching universes, non-locality or retro-causality.

The big question for me is basically: Why do people have such a hard time accepting determinism? Some will object and say "oh but it's SUPER-determinism", but that makes no sense whatsoever, either everything is determined (determinism) or only somethings are (quasi-determinism). By accepting it we escape Bell's theorem. Bell himself was well aware of this and mentioned it a few times in interviews in the 80's.

I know some are concerned that if we accept complete determinism we can no longer do science because it's all a HUGE CONSPIRACY. Gerard 't Hooft has answered critics who bring up this here: arXiv:1112.1811, section 6.

Additionally, another author who has taken a deeper look into the worries of "conspiracies" in QM interpretations is Peter J. Lewis in his paper "Conspiracy Theories of Quantum Mechanics" (also here).

So to all of you who dismiss local hidden variables I ask: Why?

Answer 1

The idea of my latest paper is simple. I experienced in other blogs that most people refuse to go with me all the way. I'll give my argument step by step and you may choose where you want to step out.

1. Consider superstring theory, in its original, completely quantized version. Many people believe it might have something to do with the world we live in. It has interesting low energy modes that show some resemblance with what happens in the Standard Model: fundamental fields for particles with spin 0, 1/2 and 1, as well as gravitons for the gravitational field, as gravitinos. The theory is not universally accepted, but it is an interesting model with many features that look like our world. Certainly not obviously wrong, and certainly very much quantum. There is a Hilbert space of states. I only use it as a model to illustrate my ideas. But step out here if you want.

2. The transverse coordinates of the string form a simple integrable quantum field theory on the string world sheet. This integrable system has left-movers and right-movers, forming quantum states, the string excitations. Now, I discovered a unitary transformation that transforms the basis of this Hilbert space into another basis. In QM, we do this all the time, but what is special in the new basis is that it is spanned completely by a set of left-movers and right-movers that are integer-valued, in units whose fundamental length is $2 \pi \sqrt{\alpha^\prime}$. Thus, we have operators taking integer values, and they are all commuting. What's more, they commute at all times. The evolution operator here translates the left-movers to the left and the right-movers to the right. Intuitively, you might find that the result is not so crazy: these integers are of course related to particle occupation numbers in quantum theory. I still have Hilbert space, but it is controlled by integers. If you don't like this result, please step out.

3. Do something similar to the fermions in the superstring theory. They can be transformed into Boolean variables using a Jordan-Wigner transformation. The superstring theory of course has supersymmetry on the world sheet. That does not disappear, but does become less conspicuous. Also the fermions are transversal. The Boolean variables also commute at all times. Next stop.

4. Realize that, if Nature starts in an eigen state of these discrete operators, it will continue to be in such an eigenstate. There is a super-selection rule: our world can't hop to another mode of eigen states, let alone go into a superposition of different modes. Thus, if at the beginning of the universe, we were in an eigenstate, we are still in such an eigenstate now. Step out if you want.

5. I can add string interactions. My favorite one is that strings exchange their legs if they have a target point in common. This is deterministic, so the above still applies. This is a stop where you may get out.

6. Rotations and Lorentz transformations. To understand these, we need to know the longitudinal coordinates. The original, completely quantized superstring tells you what to do: the longitudinal coordinates are fixed by solving the gauge constraints (both for the coordinates and the fermions). The superstring has only real-number operators, of course non-commuting. This step tells us that only 10 dimensions work, and fixes the intercept a. Don't like it? Please step out.

7. What I have here is a Lorentz invariant theory equivalent to the model generated by the original superstring theory, but acting like a cellular automaton. It IS a cellular automaton. Any passengers left?

Answer 2

The idea of superdeterminism is not really about free will. Free will is a concept that is very hard to define in a logical-positivistic way. If you don't believe me, try to define it! If you can't say exactly what you mean by a notion, in terms of "If I do this and that, what happens?" then it is not clear that the notion is well-defined. There are many questions which are just your brain fooling you into seeing sense where there is none.

First, I want to say that 't Hooft's original ideas were profoundly nonlocal and would not resemble the local cellular automata ideas of, say, Wolfram. In recent years, 't Hooft has considered the idea that there is a deterministic theory that is local in space-time which reproduces quantum mechanics. This idea is clearly wrong, and 't Hooft is talking nonsense.

't Hooft's old ideas of an underlying realistic theory were not so silly, since they came on the heels of the holographic principle. Once you realize that gravity is defined far away on a holographic screen, the idea of hidden variables becomes more plausible, because the physics of gravity is nonlocal in a way that suggests it might fix quantum mechanics. There is no real proposal for doing this, however, just vague speculations.

Holographic hidden variables could conceivably even be holographically local, meaning that they are local on a holographic screen. There is very little that can be said without a precise proposal for what these variables are.

But if you want hidden variables without holographic non-locality, then you are in trouble. You are trying to get out of the problem using "superdeterminism", the idea that the polarization settings are determined in advance, and so that Bell's inequality violations do not necessarily mean that local hidden variables are logically impossible.

Superdeterminism is silly

The proof of Bell's theorem tells you that measurements of different polarizations of far-away particles have statistics that are not reproducible in advance by the electrons alone, making crib-sheets when they are close about what the answer is going to be for the different experiments.

If you want to use the superdeterminism loophole, you need to assume that there are electron crib sheets which tell the electrons how to behave, and further, there is some mechanism which links the electron crib sheet to the choice of the experimental apparatus of which direction to measure, so that the direction one chooses to measure is somehow determined by these crib sheets.

To understand how ridiculous this is---- I could program a computer to run a random number generator, and set the polarizers according to the outcome. Then whichever random number generator I choose to use, the result must be correlated in the exact same way with the crib sheets.

If I use a thermal random number generator (a heated chip which reads out random 0s and 1s), the result will have to be correlated with the crib sheets. This correlation cannot change even if I change the temperature, altering all the Avogadro's number of particle positions and velocities. It doesn't change if I touch the chip to a hot liquid, introducing new atoms. The correlation doesn't care if I flip a coin and switch out the random number generator for another one, or if I rewire the experiment to make different outcomes correspond to different polarization settings. The nature of the conspiracy is so implausible, that it requires an intelligent agency which knows exactly what I am doing, and rearranges all the crib sheets and correlations to make everything come out right.

It is just plain impossible to imagine such a mechanism. It defies common sense that for any randomization procedure one can dream up, the results are correlated with the electron crib sheets. Further, if you have a beam of different correlated electron pairs, you might measure this electron pair or that. The mechanism has to be correlated with all the electron crib sheets. I think that this is sufficiently ridiculous that to call superdeterminism a loophole is just an abuse of language--- it is a loophole in the same way that we could all be dreaming in the matrix and the aliens have set up the outcome to look like quantum mechanics is true. It's no more plausible than this sort of nonsense.

Free Will

You brought up the issue of free will, and this is an old saw from philosophy. The actual history of the idea is important--- it comes from a religious paradox:

• If God knows what is going to happen, and made it all happen outside of space and time, how can God punish people for doing evil by sending them to hell?

This is the essential question that bothered people about Christian theology, and led to free will debates. The notions in this question are very hard to define in a logical-positivistic way, and when you do define them this way, the problem evaporates. There is no problem here, and there never was, independent of the fact that the notion of God has not been properly defined, nor its properties in any way deduced from a framework which is capable of persuading anyone in any way except by force of social convention.

To make a logical-positivistic description, you have to define all properties in terms of sense experience. So one can take a definition of free will as follows:

Free will (version 1): If I have a ham sandwich and a cheese sandwich, and I place them in front of me, and I am told to "take one and eat it", then I end up holding one of the sandwiches and eating it.

This is clearly no good. The idea we have of free will is not that we do things, but that we could have done something else. So try this:

Free will (version 2): If I have a ham sandwich and a cheese sandwich, and I am told "If you disobey this prediction, you will get $1000. I predict that you will take the ham sandwich and eat it", then I will take the cheese sandwich and eat it. Likewise, if I am told "you will eat the cheese", I will eat the ham.

This definition is not so great either. It is saying that I am capable of spiting any prediction about my behavior, if I am motivated to do so. This is independent of determinism: if the universe is deterministic, like a being in a computer simulation, you can still have this type of spiting behavior. All it says is that if you predict the outcome, and then tell the person the outcome you predicted, you change the outcome, so that it cannot be predicted anymore.

But this is the closest I can see to making sense of the concept of free will. So free will for me means the following:

Free will (version 3): Given access to the predictions any algorithm that purports to predict your behavior, and gives you incentives to spite the prediction, an agent has free will to the extent that the predictions will not come true.

This is true of people: if you tell a heavy smoker "Do not smoke for a year, and you will get a million dollars", then it is likely that the person will not smoke for a year. But this is clearly nothing to do with what people's intuition about the thing is. The intuition is that the prediction can be spited even if it is behind a curtain, hidden from the agent.

But if you don't tell the agent the prediction, there is no sense in saying the agent is somehow behaving non-freely in doing what you predict. I predict that you will get out of bed tomorrow, but this doesn't mean that you don't get out of your own free will. It only means that if I tell you that I predict this, and give you a big incentive to stay in bed, like a million dollars, and you still get out of bed.

The God business at the beginning is really resolved by defining God properly, but even supposing you believe that God is an external agent that knows the future and punishes sinners in the afterlife (something which I can't make logical positive sense out of), the fact that God knows the future in this metaphysics does not mean that you didn't choose it, since God didn't tell you the prediction and ask you to spite it!

In a certain sense, actually, in many religious traditions, God does tell you some predictions about human nature and ask you to spite them--- the prediction that human beings will be cruel and capricious, for example. This type of thing is asking human beings to spite predictions about the general nasty character of human relations in a Darwinian world, and the insistent demand that one spite these predictions, despite there being no incentive to do so, is the major purpose of religious belief.

Anyway, this is a major digression. The point of this is that the concept of free will is not well defined, and any way of defining it positivistically, it is either obviously true that human beings have free will, obviously false, or obviously meaningless to ask the question. The fact that free will has no definition, or at least, no consistent agreed-upon definition, should make one pause whenever someone discusses the concept, since this person can impose whatever definition he or she likes on it, and argue from this metaphysical position.

The position that superdeterminism means we have no free will is only true in a sense that it is determinism. This is not conflicting with free will in the definition I gave above, since determinism doesn't tell you what your choice is going to be and ask you to spite it, less does it give you incentive for doing so.

One way to try to violate the definition of free will above is to send correlated electrons to distant experimentalists, Alice and Bob, and try to predict Alice's polarization settings by capturing the electrons along the way and measuring their crib sheets (imperfectly, by measuring their spins). The issue of course is that in the superdeterministic view, the person capturing and measuring the electrons is not predicting anything about Alice and Bob, because the electron's crib sheets are now correlated with this person's polarization settings, and not Alice's or Bob's anymore. This sort of nonsense makes all experimental thinking and scientific hypothesis testing impossible, and it is really a form of magical-universe hypothesis. One must reject it a priori.

So, unlike the concepts of even God and religion, I can't see any way to make sense of the concept of superdeterminism in a logical-positivist framework. I personally consider the answer to the question as a resounding "no". No, it is not true that there is a superdeterminism loophole, and local hidden variables are just plain ruled out by Bell's inequality violations.

Answer 3

The essence of Bell's theorem is that the following three assumptions cannot be true together: 1) Freedom of choice; 2) Locality; 3) Correlations require explanation (in the form of the common cause principle).

It has nothing to do with hidden variables or Realism (as Zeilinger might have argued), or Determinism, or projection postulate. These do not need to go into the assumptions.

So if you want to retain Locality and hates inexplicable correlations, discarding freedom of choice is your only choice.

But the experimental verification of Science itself hinges on the freedom of choice (in changing parameters, in preparing the desired initial states etc.). Without this assumption, all our current physical theories might be a grand illusion, and the real theory may be totally different. I don't see how 't Hooft's argument in the paper you mentioned can avoid this conclusion.

Answer 4

I would say the Bell theorem is also based on some other assumptions that can be questioned, for example, the projection postulate, which is typically required to prove that the Bell inequalities can be violated in standard quantum theory. The reason to question this postulate is that, strictly speaking, it contradicts unitary evolution of quantum theory (this is the well-known measurement problem in quantum theory). (Furthermore, this postulate directly introduces non-locality, and that may be one of the reasons why people tend to rule out local hidden variables). On the other hand, there is no loophole-free experimental evidence of such violations. Following other people, I consider this issue in an article published in the Int'l Journ. of Quantum information ( http://www.akhmeteli.org/akh-prepr-ws-ijqi2.pdf ). There is also a substantially updated version at http://arxiv.org/abs/1111.4630 .

Answer 5

After conducting a 12 year experiment to see if events are predetermined, i.e., super-deterministic as John Bell had postulated, the evidence obtained absolute and precise results to substantiate that absolute determinism exists. I recently applied this discovery to the recent Higgs boson preliminary discovery and found a fundamental omission error has been made. The findings have recently been published in a fundamental peer-reviewed physics journal:

http://fundamentaljournals.org/ijfps/downloads/37-IJFPS_Dec_2012_44_47.pdf

What this omission error has revealed is that our perception of reality is effectual based which has blinded us to true cause and effect. Physics searches for the fundamental "interaction" of our existence via the "effects" of interactions, not the cause of them. Case in point, can an interaction take place without a selection first being made?