# Dispensing with the "a priori equal probability" postulate

I find the "a priori equal probability postulate" in statistical mechanics terribly frustrating.

I look at two different states of a system, and they look very different. The particles are moving in different ways! I like to imagine that if you zoom in on the first system you might spot a dozen particles flying off, dancing along to their own chaotic 12-body problem, while in the second system 2 particles might be shooting off, trapped in a romantic orbit. I see absolutely no reason for assuming that two completely different micro-states with the same total energy will occur with equal frequency.

As far as I can see, Laplace's Principle of Insufficient Reason works great for decks of cards, because cards are all the same, except for their labels. But there's more than labelling going on with a physical system of atoms - different states are actually different!

So I want to ask about an alternative justification for this principle, which doesn't assume it a priori. I found the argument in Landau and Lifchitz, but they don't discuss how it relates to the aforementioned postulate.

We imagine that the microstate of our system at some time $t$ is governed by some probability distribution, $\rho(x;t)$ in phase space. Then the probability density function will evolve over time, but will remain constant at a point moving along with the phase-flow - this follows from Liouville's theorem and probabilistic common sense.

Now the only sensible definition for an equilibrium distribution I can see is one in which the pdf stops evolving, $\rho(x;t) = \rho(x;t+\Delta t)$ for all $\Delta t$. But this means that the pdf (which we can just call $\rho(x)$ now) must be constant along trajectories, and so is a constant of the motion.

But this means $\log \rho(x)$ is a constant of the motion. But this is additive (because probabilities multiply), and mechanical systems only have 7 additive integrals of the motion: energy, momentum and angular momentum. And if we assume Galilean relativity and say our system isn't spinning, then if the probability distribution is an equilibrium distribution, it must be a function of the energy alone (actually, to retain additivity, we must just have $\log \rho \propto \text{Energy}+\text{Const}$). And crucially, if we fix the energy, then $\rho$ must be uniform.

It seems to me this argument gives the "a priori equal probability postulate" with no major assumptions. We simply said:

1. Our system's state is governed by some probability distribution (seems reasonable).
2. This distribution, whatever it is, should evolve in time so it is constant with the phase flow (again, given Liouville's theorem this just seems obvious.)
3. This distribution shouldn't change over time (our macroscopic system at equilibrium doesn't seem to be changing, so whatever probability distribution we pick, it shouldn't change either.)

And from this we derive that our probability distribution is a function of the energy alone. If we add a fourth requirement:

1. The total energy should be constant.

Then we see the only way of satisfying all 4 of these reasonable-looking requirements is, remarkably, the microcanonical distribution. The equal probability postulate is our first beautiful theorem rather than a doubtful, unjustified assumption.

My question is, is this argument legitimate? If not, where is the flaw? And can this flaw be patched up? This postulate is so ubiquitous in statistical mechanics texts that I can't believe it can be dispensed with so simply.

• Dispensing with Insufficient Reason is not new. Do you know about E. T. Jaynes's ideas, particularly in section 2 of this paper? Commented Apr 25, 2015 at 13:25
• Your detailed argument has many assumptions that didn't survive into your final summary. For instance you had log $\rho$ (x) a constant of the motion but then said it was uniform. You claimed that probabilities multiply but that only happens when they are independent so you've simply assumed away the chaotic 12 particle dynamics as well as the two particle orbits. Commented Apr 25, 2015 at 18:19
• @Timaeus my understanding is that Liouville's theorem implies that $\rho (x)$ is a constant of the motion along a trajectory of an equilibrium distribution. But that doesn't mean that $\rho(x)$ is constant everywhere - it may different on different trajectories. But I'm trying to show that it is the same on trajectories with equal energy, which is basically the a priori equal probabilties assumption. So I see no contradiction between noting that $\rho (x)$ is a constant of the motion and then showing/assuming it is, on top of this, uniform everywhere.
– tom
Commented Apr 26, 2015 at 3:24
• @Timaeus I do agree that there is probably a problem with making $\log \rho (x)$ additive. This is something Landau did, which I didn't fully understand the justification for. If you have two independent subsystems and combine them into one system then (ignoring surface effects) I imagine you would expect that the probability of having system 1 in state $A$ and system 2 in state $B$ simultaneously is just the the product of those individual probabilities. So this situation seems to obey additivity.
– tom
Commented Apr 26, 2015 at 3:29
• But, as you suggest with the chaotic particles, assuming additivity for all subsystems is probably too strong an assumption, because you could imagine 1/2 of those particles in 1 subsystem and 1/2 in the other, and then I guess additivity wouldn't be obeyed.
– tom
Commented Apr 26, 2015 at 3:31

I can see different subtleties in Landau's argument. First of all, it isn't entirely clear what is meant by "there are only seven additive constants of motion". To give an example, consider a single particle hamiltonian: $$H=\frac{\mathbf p^2}{2m}+m\omega ^2\frac{\mathbf q ^2}{2}.$$ For this hamiltonian there are several conserved quantities: $$e(\mathbf p,\mathbf q)=H(\mathbf p,\mathbf q),$$ $$e_1 (\mathbf p ,\mathbf q)=\frac{p_1^2}{2m}+m\omega ^2 \frac{q_1 ^2}{2},$$ etc.

Note that for a system of $N$ of these particles, the quantity: $$E(\{\mathbf p _i\},\{\mathbf q _i\})=\sum _{i=1}^N e_1(\mathbf p _i ,\mathbf q _i)$$ is conserved and perfectly qualifies as an additive constant of motion. As another example, in his treatise 1, Gibbs considers the possibility that for a system composed of $n$ oscillators with $k\leq n$ different frequencies $\omega _i$, a (canonical) distribution of the form:$$\log \rho = \alpha -\beta _1 E_1 -\beta _2 E_2 -\dots -\beta _k E_k$$ applies (here $E_i$ are the energies associated to the frequency $\omega _i$, that are separately conserved).

So it seems that the whole discussion is really making the assumption that the pdf depends only on $\mathbf P , \mathbf L , E$, which may be seen as the only constants of motion which aren't specific of any particular hamiltonian.

To better clarify this point, suppose that we have two completely separated systems "1" and "2", for which is separately true that the equilibrium pdf $\rho _i$ depends only on $E_i,P_i,L_i$. Since the systems don't interact with each other, statistical independence allows us to conclude that the composite system's pdf satisfies: $$\log \rho = \log \rho _1 +\log \rho _2,$$ however the left hand side is a function of $E_1,E_2,P_1,P_2,L_1,L_2$ and so it must be $\rho$. Landau's argument, "the pdf is additive and there are only 7 additive constants of motion", if taken literally, would allow us to conclude that the pdf of this composite system "1"+"2" depends only on $E,P,L$, which is ofcourse false in this situation. Indeed, it is precisely the (additional) assumption that also $\rho$ is a function only of $E=E_1+E_2$ etc., that allows us to conclude that $\rho$ is actually a linear function of those quantities. Clearly this assumption makes sense if the systems are actually in thermal contact and if they are large enough so that the surface energy beetween the two can be neglected.

In the particular case of the microcanonical ensemble, one evident problem in the assumption that the distribution is a function only of $E$, in particular a $\delta$ function, is that there could be subregions of this surface, having a finite measure, that are never reached during the system motion. In the example of the oscillators given above, the points of the surface with $E_1(p,q)\neq E_1(p(0),q(0))$ are never reached during the system's time evolution. I honestly don't know much about the subject, but the key word here is ergodicity. The ergodic theorem states that given a measure space $(X,\mu)$ with a measure preserving time evolution map $U^t$, the necessary and sufficient condition for the time averages to exist and agree with the ensemble average (with respect to $\mu$) is that the space is metrically indecomposable. This means that if $A\subset X$ and $U^t A = A$ for all $t$, then $\mu (A)=\mu (X)$ or $\mu (A)=0$. I think that Landau is alluding to this in a footnote of §4.

To summarize, if you want to obtain the microcanonical ensemble, at some point you have to introduce some additional assumptions which are external to classical mechanics. The "a-priori equal probability postulate" is a way to do this (probably not the only), which must be taken as a phenomenological assertion regarding systems, justified by its agreement with experiment.

• Thanks for your answer! I can't find any fault with your examples of other additive constants of the motion. Frankly I don't understand mechanics well enough to see why these examples don't contradict the result discussed here, for instance: physics.stackexchange.com/q/110609
– tom
Commented Apr 26, 2015 at 5:13
• On the other hand, I was also concerned that this contradicted the existence of non-ergodic distributions :) I think (at least one) fault lies in the independence assumption used to get the additivity of $\log \rho$: if our system isn't ergodic, then it may be that two subsystems cannot simultaneously be reached, in which case they clearly aren't independent! On the other hand, I'm pretty sure the ergodic assumption is equivalent to the a priori probability assumption (though I can't see how to prove that now) so if we have to assume ergodicity then we haven't shown anything new.
– tom
Commented Apr 26, 2015 at 5:17
• One other thought. People often say that statistical mechanics can only be justified by its agreement with experiment. But the problem is that people apply this theory to all sorts of situations where experiments aren't possible. For instance, I've done work modelling argon adsorption inside mesopores using GCMC. If we're going to use thousands of computing hours to calculate integrals of statistical distributions, we want to have very good reasons for thinking they approximate reality. And just because it worked for a macroscopic system we can measure doesn't imply it works everywhere
– tom
Commented Apr 26, 2015 at 6:32
• @tom you are welcome! Indeed, my examples don't contradict the result that every system shares the same seven additive constant of motion. My point was to show that it is easy to think of a system that has other constants of motion different from these, and which may be as well regarded as "additive". Commented Apr 26, 2015 at 6:36
• About your second comment, consider that when one talks about ergodicity in this context, the distribution involved is always the microcanonical one. The ergodic hypothesis is the statement that the time averages of observables equals the microcanonical average; put this way, it is clearly equivalent (at least from the computational point of view) to the “equal a priori probability” assumption, a part from the fact that this is usually formulated in terms of a range of energy $\delta E$ which is totally irrelevant (from the physical point of view) to the computation of the integrals involved.. Commented Apr 26, 2015 at 6:52

Without analyzing your (or Landau's) argument in depth, I'd like to point you to other alternatives to Laplacian Insufficient Reason. The idea of getting rid of Insufficient Reason is not new and was one of the drivers that motivated the great probability theorist and physicist Edwin T Jaynes, who was unsatisfied with Insufficient Reason unless clear physical or other symmetries of a system motivated it. A beautifully written summary (one of the best passages of technical writing of all time IMO) of his fundamental ideas is to be found in Section 2 of this paper:

In short, he looks to the Bayesian notion of finding the most "unbiased" distribution of energies: what probability distribution leaves the maximum possible residual uncertainty about the problem, i.e. what probability distribution constrains the observed data (total energy E and mean energy per particle $\langle E\rangle$) the least so that a sequence of samples of individual particle energies from the probability distribution contains the most information, i.e. is least predictable. The answer is the microcanonical distribution. One is thus inferring least from the data.

Now Jayne's Maxent ideas are not universally accepted and there has been a great deal of work since. But my point is that you should not be shocked that there are challenges to Insufficient Reason, particularly by someone with as broad and deep outlook as Landau. Many physicists have refused to accept Insufficient Reason and sought either to find deeper justification of it, or supplant it.

• Thanks, I've read a number of Jaynes papers, including the one you reference, but I must admit I'm not sure what to make of them :) Instinctively, I want more of a 'mechanical' justification! I'm feeling now that the ergodic hypothesis really does capture the heart of the issue, except that it is too weak (we need ergodicity over reasonable time lengths, not in the limit) and too strong (we don't really need complete ergodicity, just good enough.) But I feel more comfortable, as the 'hard mathematics' category is much simpler than the 'hard philosophy' one!
– tom
Commented Apr 28, 2015 at 6:26
• @tom "'hard mathematics' category is much simpler than the 'hard philosophy' one! " I heartily agree. Probability theory is MUCH harder than most people believe and very much a work in progress: have you seen the discussions of e.g. "Chance versus Randomness" at the Stanford Encylopoedia of Philosophy? One of the questions people often ask on this site that amuses me is "Why can't QM be modelled by classical statistics", with the clear undertext that they think classical probability would be easier than complex amplitudes. My reaction to this is that QM is "easy" in the sense that one ... Commented Apr 28, 2015 at 6:31
• ... can always ask Nature for the answer in a QM problem by doing an experiment: classical probability seems mind bendingly harder to me in comparison. Commented Apr 28, 2015 at 6:33
• @tom This is the sum total of all I feel that I truly understand about classical statistics: (1) bare measure-theoretic models and mathematical proofs of their properties from axiomatic definitions of Borel algebras and measure (2) a practical notion of statistical hypothesis testing: if something comes out 6 sigma from your model's predictions, you know you're in strife (3) my intellectual capacity craps out somewhere not too far beyond (1) and (2) and I find classical probability utterly bewildering. Commented Apr 28, 2015 at 6:42
• I never thought of QM that way :) Also, this paper: sbfisica.org.br/rbef/pdf/060601.pdf is a good read if you're familiar with the basics of Measure theory, and even if you're not. It shows how ergodicity would indeed settle most disputes (it even implies the microcanonical distribution is the unique (sensible) equilibrium distribution). I was particularly surprised to find that the ergodic hypothesis & mixing has been proven for the general hard-ball model (i.e. n-dimensional billiards) which gives me a lot of confidence in it's validity or near-validity.
– tom
Commented Apr 29, 2015 at 3:05

Landau--Lifschitz is not reliable. I never recommend it to someone who does not already know the basics. It can often be inspiring....to an advanced physicist.
This "proof" if a famous fallacy. It relies on his argument that the distribution function of the combination of two independent subsystems is the product of their individual distribution functions. Now, when he argues in favour of this much earlier in the book, he explicitly points out that this is approximately true provided the interaction between all the subsystems of the total system is weak, each subsystem is small compared to the total, and their combination is also still small.
But he "forgets" all these conditions by the time he gets to the part of the book you cite. So you cannot assume that the logarithm is additive. Within certain limits and for certain purposes it is approximately additive...but not really.

There is a vast literature in the philosophy of science on how to justify the foundations of Statistical Mechanics, and Landau's line of reasoning is not accepted. I recommend Janneke van Lith (2001). Ergodic Theory, Interpretations of Probability and the Foundations of Statistical Mechanics. for a good survey of recent attempts. None are successfull (except mine ;-P)

Somewhere or other one must let the number of degrees of freedom go to infinity, or the statements of Stat Mech and Thermodynamics are not exactly true, only approximately true. His "proof" doesn't do this explicitly. So it is obviously wrong.

Dr. van Lith's doctoral thesis is open access http://dspace.library.uu.nl/bitstream/handle/1874/657/full.pdf?sequence=1

as is her review of Guttmann's misch-masch http://www.projects.science.uu.nl/igg/dis/guttmann.html