Could the measurement problem be solved by string theory / other ToE?

Question

A quick wikipedia search will reveal the many different approaches to the measurement problem in quantum theory. What I find strange is that they even make up all these weird hypothesis like true randomness, many worlds, bohm and other interpretations when there isn't any ToE they're working it out from. I'm just a layman so maybe my assumption is wrong, but wont a fundamental theory inevitably change quantum theory?

Is really a realistic model where superpositions are nothing but lack of information on our part not likely to come from the ToE?

To my layman mind it seems premature to even begin to try to answer these questions if a theory of everything isn't in place. Or is there actually a reason that people are trying to solve this now? Such as: no matter what the ToE is, quantum theory wont change, superpositon will always exist etc.?

Answer 1

We know from the successful experiments demonstrating Bell's inequality and entanglement that the universe is not classical, but quantum mechanical. This means that it behaves in very non-intuitive ways. The "measurement problem" arises because we can't formulate a good intuitive picture of what's going on. One way that we try to explain it, the "Copenhagen interpretation," makes calculations relatively easy, but has a mysterious step, "measurement," which eludes our intuition. Explaining this measurement step is known as the "measurement problem," and since it is fundamentally a problem with our intuition, there may be no solution (note that adherents of the many-worlds interpretation and the de Broglie-Bohm pilot-wave interpretation claim to have solved the measurement problem, but in each case they have replaced it with something equally abhorrent to our intuitions).

Quantum mechanics isn't just a theory of the universe, but really a framework used for constructing all sorts of theories (the Standard model, string theory, non-relativistic quantum mechanics). For a theory of everything, there are two possibilities. The first is that it's still fundamentally quantum mechanical, in which case the situation will remain much the same and the measurement problem will not go away. The second is that it is fundamentally something other than quantum mechanical. Because it still has to approximate quantum mechanics, it's not going to behave according to our classical intuition. I think this means that it would very likely be even more non-intuitive than quantum mechanics, and the measurement problem would be replaced by something even more difficult to understand.

Answer 2

Dear Qurious, for all practical and experimentally testable purposes, the measurement theory was understood in the late mid 1920s, soon after quantum mechanics was discovered, and the "measurement problem" was solved at the level of phenomenology. The old interpretation of quantum mechanics left some questions open but from an empirical viewpoint, they were "academic questions" only.

Later progress, especially the derivations of decoherence in the 1980s, has confirmed the Copenhagen picture and derived the classical-quantum boundary, a missing piece in the old Copenhagen approach, from the rules of many-body quantum mechanics itself. All problems related to the measurement theory have been answered in principle.

People may prefer different philosophies to interpret quantum mechanics - consistent histories; many worlds; the old positivist Copenhagen interpretation - but they ultimately agree about the predictions for all experiments so their philosophies are physically equivalent. What's important is that the predictions may be made for every situation now.

So the measurement problem can't be solved by quantum field theory, string theory, or anything else, because it has been solved for a long time before these newer layers of physical knowledge started to be studied.

Quantum field theory, string theory, theories describing states of condensed matter physics, nuclear physics, atomic and molecular physics, optics, and other subdisciplines of physics that depend on the quantum phenomena take the universal postulates of quantum mechanics as exactly valid principles that cannot be modified in any way.

Modern developments in the research of entanglement, original started by Einstein and his collaborators (especially Bell's inequalities, GHZM state, and Hardy's paradox), have eliminated all doubts that the new quantum description of the reality has to be taken seriously and all "classical" models of the quantum phenomena may be falsified.

So string theory - and even quantum field theory - don't change anything whatsoever about the shared framework of quantum mechanics and its postulates. The question whether our Universe follows the known laws of quantum mechanics has been answered, de facto for 85 years, and most likely, nothing will ever change about these basic matters again.

There is no real problem over here and the postulates of the quantum dynamical framework below will stay with us.

1. The world is associated with a Hilbert space
2. Every complex linear superposition of two allowed vectors is allowed
3. Every observable is associated with a linear Hermitean operator
4. The squared absolute values of probability amplitudes - complex inner products with eigenvectors of the observables - determine the probabilities that one or another result will be observed
5. Nothing except for the probabilities may be predicted; it is incorrect to imagine that any physical system that could be affected by quantum mechanics has any well-defined, determined properties (variable quantities) prior to the measurement
6. Evolution is encoded in the Hamiltonian or the action, via the Schrödinger's equation, Heisenberg equations of motion for the operators, Feynman's path integral, or another equivalent mathematical formalism

What quantum field theory and string theory are doing is to find and investigate the more correct Hilbert spaces and the "Hamiltonians" (or equivalent structures) acting on them that define the dynamics. There are also people who study theories based on the basic assumption of "denial of quantum mechanics" but they have nothing to do with research in particle physics or string theory.

Of course, it is hypothetically conceivable that sometime in the future, one (or several) of the postulates will be falsified experimentally. If that occurs, quantum field theories, string theory, and many other theories in nuclear, condensed matter, and other branches of physics will be showed invalid or incomplete, too. But the "fear" that this could happen doesn't mean that science shouldn't build on the assumptions that seem to be valid according to the current state of the art. If the "fear" were enough to stop research, science would become impossible.

There is nothing "premature" about taking the principles of quantum mechanics as established facts, especially because they have been known for 85 years and every single experiment that has been done in those 85 years has confirmed that they are correct.

In his excellent 1994 talk Quantum Mechanics In Your Face, eminent physicist Sidney Coleman compared the resistance to quantum mechanics to the resistance to the heliocentric theory. People used to say that the geocentric beliefs (much like the belief that the world has to be classical) existed because it "looks like" the world is geocentric (or classical). How would it look like if the world were heliocentric (or quantum)? Well, it would look like the real world. Welcome home. By the way, Coleman's talk was given 17 years ago but most people still haven't accepted quantum mechanics as a fact.

Answer 3

I am optimistic that string theory in its final form will explain quantum mechanics.

The simplest way for this to occur would be for the final theory to be intrinsically quantum; not a "quantization" of a classical theory, though it must still have a classical limit, an approximation which behaves classically. This would mean that quantum mechanics and the final theory were a package deal, mathematically.

However, I mean a genuine explanation. The truly problematic feature of quantum mechanics is not nondeterminism, it's antirealism. If quantum mechanics were a classically random theory, we would have a picture of what goes on between measurements, e.g. a "Brownian-motion-like" evolution of physically localized properties. But quantum probabilities arise from complex numbers. This allows destructive interference to occur: the contributions of histories to the path integral can cancel. This cannot happen in classical probability theory, where probabilities are always nonnegative and thus always add in a nondecreasing way, and it is the main obstruction to a realist account of quantum mechanics. A classically probabilistic mechanics that reproduces quantum mechanics - like Edward Nelson's stochastic mechanics - violates the spirit of special relativity, because it involves a random walk in all directions of configuration space. A nonlocally deterministic mechanics like Bohmian mechanics is also conceptually nonrelativistic. So instead we hear that things don't have properties until they are measured. That's "antirealism".

Here is an example of a third realist approach to quantum mechanics, and how it might be realized by string theory.

There is a similarity between quantum field theory in imaginary time and classical statistical mechanics at finite temperature that has been noted many times. This might suggest an explanation of quantum probability in terms of classical probabilistic dependencies which are local in space and time, but which give rise to spacelike correlations via a zigzag in time - forward along one lightcone, backward along another. There are a number of proposed formulations of quantum mechanics which have this aspect of bidirectional causality, but none of them provides a rigorous derivation of the quantum framework from ordinary probability theory.

One way that might happen is if you were calculating probabilities on a space-time manifold that was not time-orientable. If you had an initial condition, a final condition, and a classical probability distribution over the number of closed timelike curves in between, maybe you'd end up with something like the transition probabilities of quantum mechanics, even with a locally deterministic law of field propagation on the manifold. But planck-scale time loops involve very high curvatures, and so in a string theory framework we'd be talking about the heavy degrees of freedom of the string. Anyway, the idea is that the fundamental theory is "classical" in the sense of offering an objective description of nature, and that the peculiarities of quantum mechanics arise simply because you're doing probability theory on a manifold whose topology is not globally determined.

Meanwhile, if I look at what's actually happening in string theory, there are formal developments which could be the beginning of movement beyond quantum mechanics. In promoting the recent work on twistors in gauge theory, Arkani-Hamed says it offers not just emergent space-time but also emergent quantum mechanics, because unitarity is not assumed. And Witten had a paper last year deriving nonrelativistic quantum mechanics from topological string theory.

However, Witten was still using path integrals, and it could be that the work of Arkani-Hamed et al is also just quantum mechanics from another angle, rather than a glimpse of something deeper. And what I wrote about quantum mechanics arising from classical probabilities that are bidirectional in time could also be wrong. But it helps to have a few such possibilities in view, when opposing antirealism.

Even if some form of Bohmian mechanics were the only viable realist interpretation of quantum mechanics, I think we would be rationally obliged to favor it over antirealism. What I actually expect, however, is that in the process of formulating the final physical theory, the incremental development of new quantum formalisms (see the end of point 6 in the list by Lubos), carried out for technical rather than philosophical reasons, will ultimately produce a version of the quantum framework where you don't need to talk about measurements and observables at the fundamental level. There will simply be relationships among things that exist according to the theory. And at that point antirealism will no longer be necessary.

Answer 4

The "measurement problem" connected with QM indeed gains much of its longevity because of the randomness that QM introduced into a fundamental physics that had previously been successful within its domain and (seemingly) deterministic. Physicists had worked with Statistical theories (where the statistics did come from ignorance of details) before, however, and have learned to adapt as none of this Interpretational debate affects the basic equations (like Schrodinger's).

A key part of the question is "wont a TOE/fundamental theory change quantum theory?". A simple answer is that we dont know: it might, but it might not touch quantum theory - it will have other work to do. A more subtle aspect of this is that in physics when a replacement theory comes along it has to respect the successes (gained over many decades or centuries) of its predecessor. As quantum theory is quite mathematical such a new theory also has to "respect" many of the theorems now associated with quantum theory as well as the experimental results. It is this factor which makes discussion on this topic lengthy: as some theorems (but not all such theorems) occasionally get finessed. Example theorems include: Bell's Theorem, Kochen-Specker Theorem - now Kochen-Specker-Conway. Conway has recently (controversially) introduced a concept like "free will" to explain a theorem!

These type of theorems all incorporate the sort of properties and issues of superposition that the question mentions.

Finally one (perhaps even quite likely) possibility is that the TOE will be just a set of (experimentally validated) equations. Then interpretational questions may still remain. No doubt modified by the knowledge of exactly what equations work, but still subject to (what will then be described as philosophical) debate.

Answer 5

"I believe in determinism ..."

What a determinism can be while dealing with a very complicated thing, with many degrees of freedom? Determinism arises when you make average of some reality.