What aspect of quantum mechanics forces probabilities to be (conventionally, at least) central?

Question

I understand how to compute probability distributions and expected values and such from quantum states, but a lot of treatments of QM make it look like this is what the wavefunction is essentially for. To me this seems similar to assuming that energy exists to describe temperature: taking a large-scale emergent phenomenon as central simply because we were aware of it first. What aspect of quantum mechanics makes it natural to say that probability and 'non-determinism' should take a central role?

In particular I don't see any contradictions or ontologically unusual things that arise from treating a wavefunction as a real, deterministically evolving matter wave, with collapse arising from considerations of large-scale quantum statistics (a subject with which I am admittedly unfamiliar). Probability is simply a matter of ignorance (i.e. of the underlying quantum state of the environment/measuring device), as usual. Uncertainty principles are an obvious requirement for any wave theory, and non-commuting observables simply arise from this and the fact that we can no longer pretend to measure a system and leave it intact. If this is correct then I have absolutely no idea why people are confused/conflicted about quantum foundations, which is why I suspect I'm missing something. If simple wave effects are what 'non-realism' (and the lack of definite position, momentum etc.) amounts to, I see no reason why anyone would be even remotely uncomfortable with it.

Now I don't want the question to seem vague/philosophical, so I'd like to point out that I'm interested in experimental results (gedanken or otherwise) or uses of/proofs in QM that make the assumption that wavefunctions are really about probability natural, rather than 'probability density as modulus' just being a feature that wavefunctions happen to have.

Answer 1

I am not entirely sure what you are asking, but since you seem to be sincerely interested in understanding some of the fundamentals of Quantum Mechanics, I'll do my best to answer what I think you are asking.

The answer to why we don't consider a wave function to be a "real, deterministically evolving matter wave" is simply that such an interpretation isn't borne out in experimental data. There are an abundance of experiments which have validated and re-validated the Copenhagen Interpretation, so you will be hard-pressed to figure out a way to explain their results while rejecting one of the bedrock assumptions of their theory.

Another good reason to avoid thinking about the wave function itself as being physically relevant is that it is not real-valued. Schrödinger's Equation is not the Diffusion Equation no matter how similar they look. The solutions to Schrödinger's Equation are implicitly complex-valued, so what's the fix: just throw out the imaginary part? Be careful: when you solve the Time-Dependent Schrödinger's Equation your energy eigenfunctions get multiplied by time dependent phases of the form $e^{-i\frac{E_n}{\hbar}t}$. What happens when the phase of the wave function is pure imaginary? Is the particle nowhere to be found?

Finally, your question may have been asked before by some very well-known physicists. Take a look at the EPR Paradox. The basic idea is that quantum mechanics implies entanglement, which seemingly violates causality: How can something I do to a particle here change another particle 1000 lightyears away? Well, experiments have actually show that entanglement is true and, therefore, that quantum mechanics is non-local. This result culminated in Bell's Theorem.

The upshot Bell's Theorem is that quantum mechaincs is necessarily non-local and probabilistic. No "hidden variable" theory can ever adequately explain its predictions. Take a look at the Quantum Eraser and Delayed-Choice Quantum Eraser experiments. They incorporate some fairly simple tweaks to the well-known double-slit experiment that help to highlight just how counter-intuitive (but true!) quantum mechanics actually is. Both of those "eraser" experiments are not merely thought experiments, either: they were actually done.

Answer 2

"What aspect of quantum mechanics makes it natural to say that probability and 'non-determinism' should take a central role?"

The fundamental reason is philosophical and comes from the Teleportation thought experiment, which raises the question of what happens to subjective experience after an hypothetical teleportation device avoids deleting the original copy: are there suddenly two versions of you? the only reasonable resolution to the thought experiment is that subjective experience becomes intrinsically nondeterministic, and that you can posit only a probabilistic certainty of being in a given copy

Why is the teleportation thought experiment relevant to quantum mechanics? because in quantum mechanics, the observer (described as a wavefunction) that interacts with a simple two-state system in quantum superposition, will become entangled, and become a superposition of two tensor product states; basically two copies of the observer, where each copy measures a different eigenvalue

This is why nondeterminism is not intrinsically related with quantum mechanics, but is related to a much deeper philosophical issue related with the subjective experience of observers. Once you understand this foundational fact, quantum mechanics stops being this mysterious theory, and it becomes something easier to intuitively understand

Answer 3

Imagine you are a single celled organism in a deterministic universe. You are asked to predict your future experiences. so you write an accurate PDE and note that some time in the future you divide and your now two selves go their separate ways. What future should you predict? If you note that many many splittings happens you can talk about the overwhelmingly likely experience and what statistically spread of various experiences it has. But this is just a deterministic universe. However you are essentially asked about a singular experience out of the whole of future experience.

So the question was to blame.

The same happens in quantum mechanics. You have a wave function, and the math is deterministic. But the math clearly indicates a splitting of the wave. A splitting of the wave into parts each of which has factors that are an eigenstate in different eigenspaces. And just like the split organism, sometimes they go their separate way and have a singular experience.

When you ask about that singular experience you bring in the probability. It wasn't there when you took the big view of the whole wave. But when you ask about the view of a separate part then you can't speak deterministically about a particular singular experience. And must fall back to a likelihood of a statistical spread of specific experiences of split waves.

Answer 4

If you think in plain simple terms, probability arise from the discrete nature of interactions between entities. This is the real deal which turns everything so strange and interesting.

Suppose you have 3 entities A, B and C, where A is the source of some perturbation to be sent to B and C at the 'same time'. Let's think of the perturbation in practical terms (i.e.: money) and assign it a unit of measure (dollars).

Now how would A send 2 dollars total to both of them?

Well, A should give them 1 dollar each and problem solved!!!. However, there is a constraint here and that is: Interactions are only done with minimun currency!!!'.

With that in mind, how can A give B and C one cent (minimun currency) at the same time? It can't!!!

At each time (interaction) A must choose between B or C to give away every cent until completes the 2 dollars to both of them. And if you think a little bit about it, you realize that the only objective solution for A must be to throw an imaginary coin each time to decide whom will receive the 1 cent!.

In the analog world of classical mechanics, A would send an infinite small amount of money to both of them (no minimum currency constraint) and what we will see is a beautiful continuos growing of B and C money pockets. No need to deal with probabilities!!!!.

This is just an easy bed time reasoning from a layman. Take it with a grain of salt.