On a measurement level, is quantum mechanics a deterministic theory or a probability theory?

Question

Quantum mechanics is a non-commutative probability theory. As such, it fundamentally behaves differently from classical probability theories. This manifests itself most pronouncedly in the uncertainty principle: a given state can not be assigned a definite position and a definite momentum at the same time.

Enter the measurement: If I understand correctly, when performing a measurement the outcome is a definite result on a classical level. I.e. once we have measured say the position of a particle, the information about where it was is saved in some classical way, where the classicality here emerges through having a large enough system.

To understand this apparent disparity, the concept of wave function collapse was regarded as the solution for a long time, e.g. as part of the Copenhagen interpretation of quantum mechanics. Nowadays it is widely accepted that there is no such thing as collapse, instead the quantum state of the universe evolves in a unitary way (i.e. by the Schrödinger equation). The apparent collapse is then explained as a result of interactions in many-particle systems (see e.g. on SE: this and this, and links therein. In particular this.). This also explains how some many-particle systems may well be approximated as classical and can store the information of measurement outcomes.

The question: Let us suppose we know the complete quantum state of the universe (or a closed system for that matter, to address StéfaneRollandin's concerns that the universe's quantum state may be ill-defined). Can we predict measurement outcomes in the future? Or can we only assign a classical probability? To reformulate: Is quantum mechanics on a measurement level a deterministic theory or a probability theory? If it is the latter how can this possibly be consistent with unitarity as described above? And are the probabilities associated still part of a non-commutative probability theory?

Note that in this question, interpretations of QM will not play a role, since by definition they yield the same theory in terms of observable quantities. I would therefore appreciate if an interpretation free answer could be given.

Answer 1

Is quantum mechanics on a measurement level a deterministic theory or a probability theory?

Probability theory. Evidence: when physicists do quantum measurements they find the results of individual runs are unpredictable. Only frequencies of multiple runs are predictable and match the theoretical results of quantum mechanics.

How can this possibly be consistent with unitarity as described above?

During a quantum measurement (measuring a system S by an apparatus A) the complete system S+A viewed at the microscopic level undergoes unitary evolution. During that evolution the system S become entangled with the apparatus A. However, by experimental design, this entanglement when viewed as a macroscopic approximation is seen to have some simplifying features:

a. The apparatus is in a mixed state of pointer states
b. The possible eigenvectors of some observable of S have coupled to the pointer states
c. Off-diagonal "interference" terms have become suppressed by decoherence due to the many internal degrees of freedom of A.

Owing to the special nature of these pointer states of A (from OP "some many-particle systems may well be approximated as classical and can store the information of measurement outcomes") we now have an objective fact about our universe.

Only one of the pointer states has in fact actually occurred in our universe (we can make this statement whether on not a physicist actually reads the pointer and discovers which universe we are actually in).

We can then make the inference that for this particular run of the S+A interaction, the system S in fact belongs to the subensemble giving rise to the occurance of this pointer state. We can make this reduction of the original ensemble based on this objective information about our universe. Restricted to this subensemble, we still have unitary evolution when viewed at the exact microscopic level.

Disclaimer: I don't know whether this really makes any sense, but this is what the reference referred to by OP seems to be saying.

Follow up question: so can we say QM is a probability theory for practical purposes but deterministic in principle?

No I think not. Here is the confusion: having banished the need for explicit wave function collapse from the QM formalism it seems that all we are left with is deterministic unitary evolution of the wavefunction of our closed system. Hence surely QM is deterministic. But no. The indeterminism in the outcome of measurements is still present in the wavefunction.

In fact the QM formalism tells us precisely when it is able to be deterministic and when not: it is deterministic whenever the quantum state is an eigenvector of the operator related to the measurement in question. Remarkably from this one postulate it is possible to derive that quantum mechanics is probabilistic (i.e. we can derive the Born Rule).

Explicitly, we can show that it is deterministic that if the evolution of S+A is run $N$ times (with $N \rightarrow \infty$) then the frequencies of different results will follow precisely the Born rule probabilities. However for a single run there is no such determinism. For a single run it is only determined that there will be an outcome.

This approach to QM is described by Arkani-Hamed here.

Edit
For a more advanced discussion of these ideas I recommend Is the statistical interpretation of Quantum Mechanics dead?

Answer 2

First of all, what physicists call the universe is the smallest truly closed system that contains us, since anything less than the whole universe interacts gravitationally with the remainder, hence is not a closed system.

Second, it is impossible to avoid at least some rudiments of interpretation since on a fundamental level, where the whole universe is considered as a quantum system, the definition of what constitutes a measurement result must be given in terms of this system, and this is necessarily a matter of interpretation.

In the thermal interpretation of quantum mechanics, which is the only interpretation that I can fully defend rationally (see also this answer), a macroscopic observable is a Hermitian operator defined as a mean of zillions of microscopic operators. All macroscopic observations are about expectations, not about eigenvalues.

A measurement result is the expectation value of a macroscopic observable, readable from some measurement device, that is nearly constant over a length of time sufficient to make the reading with the requested accuracy. (For example, as long as the pointer heavily oscillates, no reading, hence no measurement is possible. The higher the requested accuracy the more constancy is required.) This is the only interpretation-dependent input to the answer to your question, but it is the common sense embodied in the statistical mechanics of macroscopic systems.

If the state of the universe (i.e., the time-independent density matrix $\rho$ in the Heisenberg picture) is known to infinite precision and the expression for the observed quantity (the macroscopic operator $X$) is known to infinite precision (e.g., one knows precisely which atoms contribute to the mean pointer position) then the deterministic Heisenberg dynamics of the observed variable defines $X(t)$ at every time to infinite precision, so that the expectation $\langle X(t)\rangle$ = Trace $\rho X(t)$ of $X$ at time $t$ is also known to infinite precision. This holds at least if an ideal computing machine is available that can calculate exactly and sufficiently fast with arbitrary real numbers. Thus in principle one can tell for sure whether the near-constancy condition for a measurement is satisfied at any given time, and one knows for sure what the measured value will be. Thus, in theory, physics is deterministic.

On the other hand, it is clear from the laws of physics such a computing device can never be built, and the required knowledge can never be collected since an infinite amount of storage is already needed for representing a single transcendental number numerically. Therefore, in fact, one has to be content with approximations, resulting in a probabilistic prediction only. Thus, in practice, physics is probabilistic.

Note that both conclusions already hold classically, with the expectation dropped.

Edit (February 28, 2019): Without having to introduce any change in the formal apparatus of quantum physics, the deterministic dynamics of the complete collections of quantum mechanical expectations constructible from quantum fields, when restricted to the set of macroscopically relevant ones, already gives rise to all the stochastic features observed in practice. I now have three detailed papers on this, and a dummy paper containing just the abstracts (of these plus a fourth one not yet quite finished):

Answer 3

Is quantum mechanics on a measurement level a deterministic theory or a probability theory?

If we know the quantum state of a (ideal) closed system, then we have the probability distribution of measurement outcomes for any of its observables. The state evolution is deterministic, measurement predictions are probabilistic. The probabilities here do not depend on the environment, notably they do not depend on the measurement process itself (but of course, for a real measurement, one must also account for classical noise factors, mixing in some extra classical experimental indeterminism).

How can this possibly be consistent with unitarity as described above?

That there is two rules driving the evolution of a quantum state, a deterministic one applying to the isolated state, and a probabilistic one describing its measurement, is the measurement problem.

See Laloë (2004) for a comprehensive overview.

Unitarity is the conservation of the overall sum of probabilities for a measurement outcome. It is precisely because measurement is of a probabilistic nature that unitarity is needed.

Are the probabilities associated still part of a non-commutative probability theory?

Observables are the subjects of non-commutativity. Probabilities are just plain ordinary probabilities.

Considering observables as random variables lead to fields like quantum logic where non-commutative probability is used. But here one could argue we are in the realm of QM interpretation, which the question (v4) explicitely requests to avoid.

Answer 4

If the universe [or any closed, perfectly-isolated system] is in a pure state, then it will deterministically evolve as a pure state.

But generally, thanks to decoherence, that one pure state consists of several "branches" (also called "einselected sectors", or "Everettian worlds" etc.) that do not and cannot ever interact with each other, because they are macroscopically different along gazillions of different degrees of freedom.

In physics and in everyday life, we don't care what the "wavefunction of the universe" is. (It's unobservable anyway.) We only care about the branch of the wavefunction that we find ourselves in.

And that is not deterministic. It is probabilistic.

Collapse has always been an essential part of QM and still is. "Collapse" is when you discover what branch of the wavefunction you are in. This type of "collapse" is consistent with unitary evolution, and indeed a consequence of it.

Answer 5

What is fundamental is the underlying equation, not the evident interpretation. Due to symmetry of this equation there exists certain conserved quantities and to each of these there is a corresponding symmetry. For example, a well known example is that time translation invariance conserves energy. However, a more interesting and stronger example is the space translation and gauge symmetries which conserve probability current and probability density. This is why probability has a such high status in qm. It is as fundamental as energy. But I think most try to lift probability to a higher level than it deserves, after all, it is just a conserved quantity.

Now, measurement level... Well, the question wanted to avoid interpretations but asked about measurement. I will try:

A particle with given energy can decay via various pathways but these are all governed with energy conservation on macroscopic level (that the energies of asymptotic states of the decay products must accumulate to the original energy). An outcome of a measurement will similarly decay the probability density into distinct worlds (replace world and co with your favourite interpretation here), but the probability density is conserved. Some processes are discrete like spin, and then similar arguments apply to integrals of probability density with perhaps some other invariances and that is what is commonly referred as probability.

Now, quantum mechanics is as deterministic as given by the Lagrangian it is governed. But I assert that the probabilistic interpretation is emergent. It just happened to be (for a very good reason) that the square of the underlying amplitude is fundamental and the smallness of $\hbar$ makes the emergence of our probabilistic interpretation even more evident. That is, of course, that macroscopically most systems behave probabilistically as the cross amplitudes fade due to de coherence and then the sum of amplitudes squared will end up as probabilistic (stochastic) process.

Answer 6

I can’t say that I can agree that one can think of QM without thinking about its interpretation when you are bringing in notions such as non-commutative probability.

Probability is something that requires interpretation. Merely to note a formal resemblance and to call quantum amplitudes non-commutative probabilities is merely eliding the problem.

As for your assertion that QM is ‘widely accepted’ to be deterministic. By whom may I ask? The links you provide are not to peer reviewed publications and two of them are to posts on SE.

Answer 7

It has to be deterministic at core. But the workings at that level are so minuscule and random that it is next to impossible to assign a cause to individual events. Probably it will be never possible to do so, but I am sure we will be able to prove that the cause exists. We will not be able to formulate it, that is another story. In fact there is a lot of effort (experimental and mathematical) to prove it is non-deterministic. If even a tenth of effort was spent on proving the causality, it would already have been succeeded. You can guess this from the kind of resistance and discouraging you observe when you even start talking about it.

In the form of probabilities, basically we are assigning averages. And averages certainly work. The fact that averages work, itself tells you that things are deterministic at core.

The fact is that certain processes, and certain scales are so subtle in nature that assigning cause to individual events is just not possible with current understanding. As the scale goes smaller and smaller, it becomes harder to assign causality and we fall back on average/probabilities.

As a very crude example even in classical physics, the water of pool evaporates over several months. Even though we can say how much water will evaporate in a day, we can not say which molecules will evaporate which day/time. The randomness prevents us from saying so. That does not mean evaporation process is non-causal at core. The computations would be just impossible to do. Quantum processes can be many times more minuscule and random.

In a coin toss experiment, we know that the outcome of first toss does not impact outcome of subsequent toss(es). We can not say this with certainty about quantum measurements. So, coin tosses are independent events. Are quantum measurements independent over time? This has not been scrutinized enough.

Due to subtleties and randomness, we can not assign the cause to individual events, it is the inadequacy, it should not define the process as non-deterministic. So there is a limit we can assign the cause, does not mean the cause is not there.

Physics is bound to hit that limit/inadequacy at some point as we go smaller and smaller. But problem with QM is that it claims there is no cause. It would be more honest if it said - yes there is a cause, but we can not formulate it. Only thing we can formulate at this time is averages/probabilities.

With time, the QM community has come to believe in it so much that it does not even want to try to establish the causality. I am not smelling a conspiracy here. They genuinely and honestly believe in it and defend it so. Reason - quantum mathematics is very accurate and convincing. Accuracy does not always translate into reality.

It made me laugh the other day when someone claimed entanglement correlations can not be causal because Bell’s inequality does not allow it. But such people fail to realize that if there is a cause, then Bell’s inequality does not even apply to this phenomena, and it is bound to be violated anyway. Something causes it to be violated - find that damn thing. Have you tried enough? If yes, where are the records that demonstrate this is how it was scrutinized and it failed the scrutiny. Most experiments from the onset, are aimed at proving non-determinism.

Answer 8

I restate the question, for clarity:

On a measurement level, is quantum mechanics a deterministic theory or a probability theory?

This is actually an extremely brilliant and insightful question; it gets directly to the heart of existing, and largely ignored, contradictions lurking within the heart of quantum theory.

First, i give you my answer to the question:

The answer is, "yes" and "no".

At the macroscopic, or "measurable" scale, quantum mechanics is, like (nearly*) all other scientific theories, both logical and deterministic, and indistinguishable in that respect from all the rest of physics theories; but at the subatomic scale, the scale where light, itself, used to observe, or "measure", begins to affect the object being observed, at that level, quantum theory is not only probabilistic, it also leads to logical contradictions.

The first logical contradiction is found at the transition from probabilistic to deterministic behavior, a theoretical spherical surface, or "event horizon", between the measurable, macroscopic world, and the subatomic part we cannot see or predict, and which can only be described, so far, in random, probabilistic terms. This contradiction is popularly known as Schrödinger's Cat.

https://en.wikipedia.org/wiki/Schrödinger%27s_cat

This quantum "event horizon" also leads us to the second contradiction: According to quantum theory, a person faced with two incoming missiles could, by using some hypothetical device to fiddle with probability at the quantum level, transform two incoming missiles into a bowl of petunias and a large sperm whale over a large M class planet to contemplate their brief existence (I call this theory the "hitchhiker's" paradox). Another example of this prediction of quantum theory would be that all of the atoms in your body could spontaneously disappear and reappear a few meters to your left, if enough time passes.

Needless to say, these conclusions, which are direct (albeit extremely improbable) predictions of quantum mechanics, have never been observed...

In order to fully explain the third and fourth contradictions, i need to introduce a couple of historical facts that most people may not be aware of, and many will find incredible:

First historical fact (third contradiction): Einstein never fully accepted quantum theory.

Einstein went to his death being quietly ostracized and laughed at by his peers in the physics community. His famous quote that "god does not play dice with the universe" was an attempt to explain what he considered a fatally fundamental contradiction in quantum theory: at the subatomic scale, unlike (nearly) all the previous and existing theories of science and physics, quantum mechanics is NOT logically deterministic! It is, in fact, illogically indeterministic, a.k.a. randomly probabilistic, a.k.a. unpredictable and unknowable, a.k.a. "god playing dice with the universe". Einstein thought that that fact alone should cause us to discard or alter the theory.

Second historical fact (fourth contradiction): Quantum theory can not be reconciled, or made to "fit", with general relativity.

We already have maxwell's em field equations that cannot be reconciled, or made to "fit", with general relativity, unless we assume at least four physical dimensions + spacetime, for a total of at least five dimensions (see "Kaluza-Klein" and "String" theories); why add another?

to be continued... have to mow lawn.

Answer 9

I think most people you ask will say that on a measurement-level QM is a probability theory, despite as you said the deterministic nature of the wavefunctions themselves. They will say these probabilities are irreducible.

Could this be true? Sure. But i also think that's it's a bit naive to completely rule out an underlying deterministic nature, at least at our current point in time/knowledge with many aspects of physics being unsolved. Just to be clear, we could be talking about something too complex for us humans to even begin to understand, ie. chaos functions. But like chaos functions, there is determinism, whether we understand it or not.

Anyway i don't think that wavefunctions being deterministic necessarily means measurement has to be. I think it's interesting that wavefunctions are deterministic but measurement isn't, but i don't know if it prohibits measurement from being probabilistic.