Can unitary probability theory (quantum mechanics) emerge from a lack of information about a deterministic process? [duplicate]

My initial question below appears to be unclear so I am rewording in more succinctly here. The pre-edit question remains below.

There exists two types of probability theory: 1-norm (classic stochastic theory) and 2-norm (quantum mechanics). 1-norm has been shown to emerge from the lack of information about a deterministic process (statistical mechanics). Have we proven that a 2-norm probability theory cannot possibly emerge from a lack of information about a deterministic process?

Pre-edit question:

From my understanding a ‘hidden variable theory’ is just saying that the current quantum theory is not a complete description of nature, and just like thermodynamics can be reduced to deterministic rules (statistical mechanics), the probabilistic nature of the wave function can be reduced to deterministic rules. Hidden variable theory simply says that it is possible traditional quantum theory is an emergent theory that results from lack of information about the specific state of this new 'hidden variable'.

One crucial difference to note here is that ‘probability’ in the quantum sense is very different than traditional probability. The probabilistic state evolves unitarily instead of stochastically and the traditional probability is recovered by squaring the wave function. This leads to interference and most of the oddities associated with quantum theory.

Scott Aaronson in his paper here reduces the possibility of hidden variable theory to whether we can map unitary to stochastic matrices in a way that has certain desirable properties. Isn’t this just proving that quantum theory cannot be explained with ‘traditional’ stochastic probability? Couldn’t there still be a way to reduce this ‘new’ type of probability down to deterministic rules with missing information (again just like statistical mechanics did for stochastic probability). Is stochastic probability the only possible probability theory that can result from a lack of information? If so, why?

marked as duplicate by John Rennie quantum-mechanics StackExchange.ready(function() { if (StackExchange.options.isMobile) return; $('.dupe-hammer-message-hover:not(.hover-bound)').each(function() { var$hover = $(this).addClass('hover-bound'),$msg = $hover.siblings('.dupe-hammer-message');$hover.hover( function() { $hover.showInfoMessage('', { messageElement:$msg.clone().show(), transient: false, position: { my: 'bottom left', at: 'top center', offsetTop: -7 }, dismissable: false, relativeToBody: true }); }, function() { StackExchange.helpers.removeMessages(); } ); }); }); Aug 20 at 9:20

• Due to the Bell test experiments we now know that you have to discard either locality (information traveling faster than the speed of light) or realism (there being hidden variables that determine the "apparent" stochasticity of QM)... Discarding locality would violate special relativity and therefore invalidate quantum field theory, the theory that has the most precise measurements in all of physics (over 1 part in 10 billion). Bell's inequalities don't tell us which of the principles is violated, but you'd have to discard the single most successful piece of physics so far to have determinism – S V Aug 19 at 19:52
• @SV So does Bell inequalities specifically show that unitary probability theory cannot emerge when local realism is assumed? When I read Bell's theorem it still looks like they are comparing quantum to classical probability theory. To me it just seems to prove that quantum cannot emerge from classical probability. – Jacob Schneider Aug 19 at 19:57
• When Bell did his work, he was a 'fan' of Einstein, he wanted to prove that he was right and put the EPR paradox in mathematical terms. Experiments later showed that the inequalities were violated, but the validity of the inequalities rely on the classical assumptions of realism and locality. The violation implies that one of them is wrong... Basically, you can choose if you want to keep the idea of determinism that you somehow decided is a key component of physics, or you can keep locality which is a key part of the most successful part of physics thus far, but you can't have both. – S V Aug 19 at 20:02
• I don't think you understand my question. Maybe I can edit to make it more clear. There exists two types of probability theory 1-norm (classic stochastic) and 2-norm (quantum mechanics). 1-norm has been shown to emerge from lack of information about a deterministic process. Have we proven that a 2-norm probability theory cannot emerge from a lack of information about a deterministic process? – Jacob Schneider Aug 19 at 20:09
• My knowledge on probability theory is not good enough to be able to answer that, but that could be a question better suited to the math stack exchange. I would assume that realism being excluded by Bell test experiments also include what you mention, but I am not sure. – S V Aug 19 at 20:24

Quantum mechanics does not introduce "new" probability theory. Classical probability theory works well in quantum mechanics for calculating mean values, deviations, etc. For example: $$ = \int_{-\infty}^{\infty}x\rho(x)dx,$$ where $$\rho(x)=|\psi(x)|^2$$.
• But here you use the 2-norm $|\psi(x)|^2$ to convert an amplitude to a 1-norm probability. You can of course claim that amplitudes are not probabilities, but that is just semantics. As Scott Aaronson discusses in his textbook on quantum computing and in arxiv.org/pdf/quant-ph/0401062.pdf the 2-norm aspect of QM seems to be very integral. – Anders Sandberg Aug 20 at 9:32
• The problem is that a state of a quantum particle if described by infinitely many parameters ($\left|\psi \right>=\sum_n c_n \left|e_n \right>$), whereas for a classical particle the initial position and velocity is sufficient. Adding finitely many hidden parameters won't help. As a consequence, some quantum states (such as superposition state) cannot be described in classical mechanics. – atarasenko Aug 20 at 11:03