What is a probabilistic physical theory?

What do we mean when we say that we have a probabilistic theory of some phenomenon?

Of course, we know from experience that probabilistic theories "work", in the sense that they can (somehow) be used to make predictions about the world, they can be considered to be refuted under appropriate circumstances and they generally appear to be subject to the same kinds of principles that govern other kinds of explanations of the world. The Ising model predicts the ferromagnetic phase transition, scattering amplitude computations of quantum field theories predict the rates of transition between different quantum states, and I can make impressively sharp predictions of the ensemble properties of a long sequence of coin tosses by using results such as the central limit theorem. Regardless, there seem to be a foundational problem at the center of the whole enterprise of probabilistic theorizing - the construction of what is sometimes called "an interpretation of the probability calculus" in the philosophical literature, which to me seems to be an insurmountable problem.

A probabilistic theory comes equipped with an event space and a probability measure attached to it, both of which are fixed by the theory in some manner. However, the probability measure occupies a strictly epiphenomenal position relative to what actually happens. Deterministic theories have the feature that they forbid some class of events from happening - for instance, the second law of thermodynamics forbids the flow of heat from a cold object to a hot object in an isolated system. The probabilistic component in a theory has no such character, even in principle. Even if we observed an event of zero probability, formally this would not be enough to reject the theory; since a set of zero probability measure need not be empty. (This raises the question of, for instance, whether a pure quantum state in some energy eigenstate could ever be measured to be outside of that eigenstate - is this merely an event of probability $$0$$, or is it in fact forbidden?)

The legitimacy of using probabilistic theories then rests on the implicit assumption that events of zero (or sufficiently small) probability are in some sense negligible. However, it's not clear why we should believe this as a prior axiom. There are certainly other types of sets we might consider to be "negligible" - for instance, if we are doing probability theory on a Polish space, the collection of meager sets and the collection of null measure sets are both in some sense "negligible", but these notions are in fact perpendicular to each other: $$[0, 1]$$ can be written as the union of a meager set and a set of null measure. This result forces us to make a choice as to which class of sets we will neglect, or otherwise we will end up neglecting the whole space $$[0, 1]$$!

Moreover, ergodic theorems (such as the law of large numbers) which link spatial averages to temporal averages don't help us here, even if we use versions of them with explicit estimates of errors (like the central limit theorem), because these estimates only hold with a probability $$1 - \varepsilon$$ for some small $$\varepsilon > 0$$, and even in the infinite limit they hold with probability $$1$$, and we're back to the problems I discussed above. So while these theorems can allow one to use some hypothesis test to reject the theory as per the frequentist approach, for the theory to have any predictive power at all this hypothesis test has to be put inside the theory.

The alternative is to adopt a Bayesian approach, in which case the function of a probabilistic theory becomes purely normative - it informs us about how some agent with a given expected utility should act. I certainly don't conceive of the theory of quantum mechanics as fundamentally being a prescription for how humans should act, so this approach seems to simply define the problem out of existence and is wholly unsatisfying. Why should we even accept this view of decision theory when we have given no fundamental justification for the use of probabilities to start with?

I apologize for the length of the post, but I would like to hear what other people have to say about this problem.

• Can you explain why does "a set of zero probability measure need not be empty" mean that you can observe an event with zero probability? I agree with the statement that a set of zero probability measure can be non-empty but IMO it does mean that our theory (which we use to produce this probability measure) predicts that these events won't occur and thus, if they do occur, we can rule out our theory.
– ACat
Jun 10, 2020 at 0:19
• Interesting question. You say "The legitimacy of using probabilistic theories then rests on the implicit assumption that events of zero (or sufficiently small) probability are in some sense negligible.", as if previously you made an argument for the latter sentence, but I could not find such argument. Or perhaps I did not understand it.
– lcv
Jun 10, 2020 at 9:33
• @DvijD.C. That is not right - the approach using the notion of predictability that Yvan Velenik outlines in his answer makes more sense, and it is in fact the criterion we use both intuitively and in practice. Merely observing an event of null probability is an entirely banal occurence: if you flip a coin infinitely many times, the exact sequence you observe is always of null probability. However, we only intuitively feel there is a problem if you have predicted this sequence in advance, not merely because the sequence is what you sampled from the event space. Jun 10, 2020 at 12:59
• @lcv To reject a probabilistic theory because some test statistic exceeds some threshold value fixed by a $p$-value only makes sense if you will agree to consider events with sufficiently small probability to be negligible, at least having fixed the statistical test in advance. Otherwise, observing a $12 \sigma$ event in a particle collider tells you nothing of any importance. Notice that you can also make any event have a small $p$-value if you pick the statistical test after observing the event rather than before, again underlining the importance of predictive ability. Jun 10, 2020 at 13:02
• OK, thanks for the clarification. I think I understand your question now but any finite string of Heads/Tails would have a finite probability. Only in the limit when you take the string to be of infinite length, the probability would vanish. I am not mathematically well-versed in mathematics to answer your question but I think this corresponds to the story unbounded operators and the fact that eigenstates of such operators are not really in the Hilbert space (rather they simply provide a basis), and thus, such states are not physically realizable.
– ACat
Jun 10, 2020 at 15:26

I don't think that probabilistic physical theories are so different from the other ones from the point of view of experimental falsifiability.

Indeed, even if your theory is deterministic, measurements are subject to uncertainties: there may be an error in the setup, human error, some external noise, etc. If your non-probabilistic theory states that something should not occur (say, neutrinos traveling faster than light) and you do observe such an event, then the first thing you do is look for such errors. Indeed, if the theory is well established, then this is by far the most likely explanation.

Now, if your probabilistic theory predicts that some event has probability very close to $$1$$ and you do not observe it, then you are certainly at least as justified to consider rejecting the theory as in the deterministic case above.

The central point is that the violation of the theory must be observable in some setting reproducibly. If the anomaly occurs just once, then you cannot conclude anything, because you'll not be able to exclude possible errors. That the theory is deterministic or probabilistic does not really matter.

Finally, concerning events with $$0$$ (or tiny) probability: such events occur constantly. If you throw a fair coin a million times, the probability of observing any particular sequence is $$2^{-1000000}$$, that is, zero for all practical purposes. But, of course, if you do the experiment then one such sequence will occur. The point is not that they do not occur, it is that you cannot predict their occurrence. In this example, there is a $$\sim 0$$ probability that if you choose a sequence before the experiment, the precise sequence you chose occurs. And if you managed to do that, then I would consider that something fishy is going on: the coin is not fair, or you are throwing it in a very special way, etc.

• The fact that measurements are subject to uncertainties doesn't change the situation, because the probabilistic theory is in principle disconnected from any phenomena, while the deterministic one is not. In fact, this is one of the reasons it's suspicious to use probabilistic methods in statistical tests of experimental errors. The problem of considering a deterministic theory to be refuted or not is the usual underdetermination problem of the empirical sciences - there are infinitely many possible theories, so why do we pick this one? Jun 10, 2020 at 10:09
• The situation with a probabilistic theory is completely different, because the theory by itself doesn't say anything about the world at all; it simply comes attached with an a priori epiphenomenal probability measure which is supposed to be in some sense physical. We can decide whether to accept or reject such a theory based on external criteria, but they don't seem to be part of the theory. I agree with your answer in practice, of course, but it doesn't seem to resolve the problem in principle. Jun 10, 2020 at 10:11
• I am not sure I understand the difference. Because of the desideratum of reproducibility, most probabilistic statements in physics can be transformed into claims that some event has probability $\cong 1$ (just by making a statement on the empirical average of specific outcomes in a sufficiently long series of experiments). Now, if a theory states that with probability $> 1 - 10^{-1000}$ something should occur and it does not, then, provided you have eliminated all possibilities of error in the experimental setup itself, the rational decision is to reject the theory. Jun 10, 2020 at 16:29
• If you do not reject the theory in such an instance, then I don't see how you can falsify any theory experimentally... Jun 10, 2020 at 16:30
• For me, whenever a probabilistic theory makes essentially deterministic claims (that is, with a probability so close to $1$ as to make the difference meaningless), it is just as good as a deterministic theory. Jun 10, 2020 at 16:31

I am not sure that the conundrum in the question (and clarified in the comments) has really to do with probabilistic theory. In my view the culprit is actually the fact that we assign physical quantities to be described by values in the mathematical entity '$$\mathbb{R}$$'. This gives rise to all sorts of paradoxes. I'll try to explain with an example.

If we drop an (uncooked) spaghetti on the floor, we will likely use the very useful assumption that the angle that the spaghetti makes with the North Pole can be described by $$\theta\in [0, 2\pi)\subset \mathbb{R}$$. Now according to the mathematical rules of probability and the concept of continuum, the probability of any given angle is zero. Yet every time we throw a spaghetti on the floor an angle is chosen (they all happen). What is the reason for this phenomenon?

If we think about it long enough the problem is that we are implicitly assuming that the nature of physical space can be described by the mathematical quantity $$\mathbb{R}$$. This is actually an heresy if you think about it. The mathematical object $$\mathbb{R}$$ has nothing to do with the physical world. It's a purely mathematical concept, that can give rise to crazy paradoxes like Banach-Tarski paradox (you can cut a sphere into a finite number of pieces and glue them back together in order to form two spheres).

If you really think about measuring the 'spaghetti angle' with a physical device, you will encounter problems when the required precision reaches the level of atoms, as then the point where the spaghetti ends become fuzzy. But the problem as you see is about the nature of space itself. This problem has to do with renormalization of quantum field theories, string theories and all that stuff but this is another story.

There are two types of probabilistic physical theory (it is not particularly important whether one takes a Bayesian approach. A Bayesian understanding of probability is helpful, because it is clearer and more accurate, but it does not much change the essentials of the mathematics).

First, in classical probability theory, outcomes are determined by unknown variables. In principle, if we have a probability destribution for the unknowns, this will give us the probability distribution for the outcomes.

Second, in quantum theory, outcomes are not deterministic. There are no unknown variables generating quantum probabilities. It should be immediately obvious that this will not give us the same mathematical probability theory as a deterministic theory. It is rather less obvious that it will give us exactly the probability theory described by quantum mechanics. Nonetheless it does. I have explained the maths in The Hilbert space of conditional clauses.

• It is in fact not immediately obvious, because the Born rule is the only probability measure you can pick for the "collapse" if you also require the measure to be invariant under unitary automorphisms of the Hilbert space. This is typically done the other way around: the assumption of unitarity is justified by conservation of probability, but one can just as easily invert this traditional argument. In that case, QM is not fundamentally probabilistic, just nondeterministic (but it still forbids measuring the eigenstate |0> of some operator measured as having value 1). Jun 10, 2020 at 10:02