Why is the fundamental postulate of statistical mechanics true?

Question

As I'm sure folks here know, the principle of equal a priori probabilities, sometimes called the fundamental postulate of statistical mechanics, states the following:

For an isolated system with an exactly known energy and exactly known composition, the system can be found with equal probability in any microstate consistent with that knowledge.

This isn't sitting right in my head because I also know that particles want to occupy the lowest possible energy.

How can it be that all microstates (consistent with the given macrostate - i.e., number of particles N, total volume V, and total energy E) have equal probability, when we know that particles want to occupy lower-energy states? Wouldn't the microstates which have more particles distributed in lower-energy states have a higher probability?

The principle of equal a priori probabilities implies to me that if we had a total energy of E = 100 J and N = 10 particles, it's equally likely to have

• 10 particles all with 10 J of energy or
• 9 particles with 1 J of energy each and 1 particle with 91 J of energy.

Whereas, to me, the second one seems intuitively far more unlikely! But each of these microstates is consistent with the overall macrostate. And the macrostate makes no statement regarding the specific energies of each particle. How do we reconcile the principle of equal a priori probabilities with the fact that particles want to assume the lowest possible energy states?

Answer 1

The "search" for lowest energy states is a transient effect. If a system is in a high energy state and can emit energy out of the system, it will do so until the energy of the system diminishes to where it is no longer energetically favorable to do so.

This "isolated system" deals with the steady state case, where energy is not entering or leaving for an arbitrarily long time. Steady state systems exhibit different behaviors than transients.

Given two microstates A and B, we know that the probability of a A->B transition occurring must be the same as the probability of B->A because both states have equal energy. It's easy to see that if the probability of those transitions are equal, then the transient case would have smoothed them out before we arrived at steady state. If P(A) < P(B), we would expect more B->A transitions than A->B transitions, by Bayes' theorem, until eventually we reach P(A) = P(B).

Worth noting however: postulates are not provable. It's called the fundamental postulate of statistical mechanics because we can't actually prove that systems will do this. However, all systems yet observed do indeed demonstrate this behavior, even systems that were designed to break it. You could, however, turn it around and say that if the micro-states are not equiprobable, then you by definition do not fully know the composition of the system.

Answer 2

The postulate of equal probabilities for each microstate is perfectly compatible with the observation that macroscopic objects try to minimize their energy.

As a concrete example, suppose I toss $2N$ coins, and I know that $N$ of them come up heads. We think of the heads as representing energy. Now split the coins into two equal piles, the first $N$ I tossed and the last $N$. How are the heads/"energy" typically distributed between these piles?

For $N = 1$, the configurations are $(H, T)$ and $(T, H)$, so one pile has all of the energy with probability $1/2$. For $N = 2$, the configurations are $$(HH, TT), (HT, HT), (HT, TH), (TH, HT), (TH, TH), (TT, HH)$$ which is already dramatically different: there's a $4/6$ chance for a perfectly even split of energy, and only a $1/6$ chance for each pile to have all the energy.

This trend continues and intensifies for higher $N$. When $N = 100$, there's still only one configuration where the first pile has no energy, but $10000$ ways for the first pile to have one unit of energy. And there are over $10^{50}$ ways to have an equal split of energy. If you do the math, you can show that it's very likely that the split is pretty close, within $25\%$. And in the real world, $N \sim 10^{23}$!

---

To put some more formalism around this, the number of ways a pile of coins can have a given amount of energy is measured by the entropy, with the total entropy of the two piles maximized when the energy is evenly split. And while every configuration is still equally likely, and it's completely possible to have a totally uneven split of energy, it is overwhelmingly more likely to have a nearly even split.

When you imagine a hot object cooling off, imagine a small pile of coins that's mostly heads, inside an enormous sea of coins that's mostly tails. If you start randomly swapping coins around, you expect the heads to disperse with overwhelming probability, even if every swap is equally likely.

There's a nice way to characterize when this process ends. For each subsystem, we compute how much energy it needs per unit change in entropy, $$T = \frac{dU}{dS}.$$ Then the entropy is maximum once each subsystem reaches the same "marginal entropy rate", i.e. $T_1 = T_2$. But our $T$ is actually the definition of temperature. We've just shown that heat flows from hot to cold, though the underlying random coin flips couldn't care less.

Answer 3

It's a design constraint rather than a fundamental truth

It's not inherently "true" so much as a design constraint when constructing descriptions of microstates.

For example, consider a description of microstates in which it is true that all of them are equally probable. Then, redefine the ensemble such that it's exactly the same, except half of the original microstates are now described as a single microstate that encompasses all of them. Obviously, they're not equally probable anymore.

So,

For an isolated system with an exactly known energy and exactly known composition, the system can be found with equal probability in any microstate consistent with that knowledge.

can be restated as:

When you're constructing a description for your ensemble of microstates, try to define them such that they're all equally likely.

Such physical models don't tend to require perfection, so it's not like everything'll instantly break if there's a small error in the description of microstates where some are a bit more probable than others.

I mean, sure, we try to fit our models to reality as best-as-possible, but some error's gonna happen.

For example, folks often talk about ensembles of coins that can be Heads-or-Tails. The obvious microstate description for a bunch of such coins is where each microstate corresponds to one set of Heads-or-Tails value for each coin. But even if we select that description, which sounds fair enough, coins don't tend to be perfectly fair, so the description won't be perfect.

A real-life scientific example of that is with chemical isotopes. This is, it's possible to ignore the differences between isotopes of the same element; this introduces some error, but in common practice it's generally allowed as a reasonable approximation unless there's some compelling reason not to.

Basically, the trick's to select microstates that're approximately equally likely. The better the fit, the more logically consistent the arguments based on it would tend to be.

Entropy tends to grow due to degeneracy

Statistical Mechanics is all about how degenerate states are more likely.

For example, an ideal gas is said to evenly disperse about a room because there're many more microstates in which it is rather than, say, all of the gas particles being smooshed together in some corner of the room.

This isn't that any single dispersed microstate is more likely, but rather there're way more dispersed microstates than non-dispersed microstates. This is:

1. Each dispersed microstate should still be as likely as any other microstate.

2. However, there're astronomically more dispersed microstates, such that they're collectively more likely than the collection of their alternatives.

However, say that you redefine the set of all states in which gas particles are dispersed about the room to be a single microstate. Then, yes, that one micro-state would be far more likely than any other. The fundamental postulate of Statistical Mechanics simply recommends that you don't define them that way, since the heterogeneity of it screws with other descriptions.

Answer 4

I think the fundamental source of your confusion is related to the following idea from statistics:

Imagine I have a coin that I flip 10 times. Intuitively, a series of flips that came out as $HHTHTTHTTH$ would be "expected", while the series $HHHHHHTHHH$ would be "unexpected". If we think about the problem carefully, however, we can see that the exact state $HHTHTTHTTH$ is precisely as unlikely as the state $HHHHHHTHHH$; they both have probability $0.5^{10}$.

The reason for the confusion is that our human brains love seeking patterns, so instead of looking at exact microstates, we tend to think about the "big picture", even when it leads us astray. So for instance, $HHTHTTHTTH$ seems more likely because there are many more states with 5 heads than there are with 9 heads. Our brain is so used to only dealing with the end results of probability that we forget that individual outcomes are not the same as the aggregate events that we care about-- mainly because there are so many outcomes of most systems that it's basically impossible to look at them individually. Time for a relevant Feynman quote:

"You know, the most amazing thing happened to me tonight... I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!"

Similarly with your example, in real life we're normally not actually concerned with exact micro states, given that each one by itself is so unlikely. So, many people intuitively consider a state where all particles have very small fluctuations away from 10 J to be the same as one where they all have exactly 10 J, even though they don't consciously acknowledge this assumption. But a state where one particle has almost all the energy is less stable with respect to small fluctuations, since the fluctuations of the many particles with little energy can add up to take a significant amount away from the particle with lots of energy.

TLDR: I think your problem is that you're unconsciously conflating individual outcomes with aggregate events.

Answer 5

The principle of equal a priori probabilities implies to me that if we had a total energy of E = 100 J and N = 10 particles, it's equally likely to have

• 10 particles all with 10 J of energy or
• 9 particles with 1 J of energy each and 1 particle with 91 J of energy.

Whereas, to me, the second one seems intuitively far more unlikely!

If the particles are identical, there is only one possible microstate corresponding to each one of these two cases, so you have a priori no reason to say that one state is more unlikely.

Intuitively, I believe that you think that the second case is less likely because there is one particle with energy far higher than all the others, and indeed there are not many microsates in which a single particles has energy far higher then the others (try to write down the possible states in which a single particle has energy greater than, say, $50$ and you will see what I mean). However, this is not in contrast with the statement that all the microstates are equally probable!

Be sure that you are not confusing the probability that a single particle has energy greater than $x$ (which is calculated over all the microstates) with the probability of a single microstate in which one of the particles happens to have energy greater than $x$...

Answer 6

You can think of this as assigning or adding energy to a random particle in the system, 1 J at a time (pretending energy is discretized at 1 J intervals, to simplify).

You have 100 J that you need to assign to 10 particles. Who are you going to give the first one to? You have no reason to prefer any at this point, so you give it to a random one.

Now you have 99 J. Who are you going to give your second one to? Here you need to consider one of two reasonable assumptions:

1. There is still no reason to prefer any particle. You continue to assign each unit of energy randomly. This is like rolling a 10-sided die, 100 times. Without doing any math, you expect each number to come up roughly 10 times. Likewise, you expect each of your particles to end up with roughly 10 J of energy. OR
2. Since the particles now have different energies, you may develop a preference. But then, do you prefer to assign your energy unit to particles with low energy or to particles with high energy? You can design some thought experiments and potentially argue for either case, depending on the system you come up, how you imagine the particles, how you add the energy...

A good system to consider is this: you have lots (billions) of particles in a small box, and you add heat to the system by putting one side of the box in contact with an external heat source. Thus, any particle in your box-system that comes in contact with the "hot" side of the box gains a unit of energy before bouncing off. If the particles don't interact with each other (they move in straight lines), then once a particle hits the hot side, it will take a relatively long time for it to reach the other side of the box and come back to hit the hot side a second time (even if its velocity increased slightly due to the energy it gained). Thus, it is likely that every other particle will hit the hot side once before this particle hits it a second time.

Of course, you can and should try to come up with your own systems to test your intuition.

Answer 7

$91J$ is less likely than $10J$, but $10J$ is less likely than $1J$.

Judging from the comments, you are making a critical oversight in your approach to the problem.

You are focusing on the fact it is extremely improbable that one particle would have 91J of energy rather than 10J of energy.

What you are overlooking is that it is also extremely improbable that nine particles would have 10J of energy rather than 1J of energy.

If we carry this heuristic argument through by plugging the numbers into the probability density function $\exp(-E / (kT))$, the two heuristic probability densities are:

$$\left( \exp(-10 / (kT)) \right)^{10} = \exp(-100 / (kT)) $$ $$\left( \exp(-1 / (kT)) \right)^{9} \exp(-91 / (kT)) = \exp(-100 / (kT)) $$

which are exactly the same.

(I assume in the above that you single out a specific particle to be the one with high energy)

Answer 8

Imagine that the system has its interactions initially turned off. In that case, why are the particles going more likely to be in one state or the other, if the state is compatible with your knowledge? If the particles interact, you can use a state that is compatible with the probability distribution of the system evolving from one of the equally likely states to where the time evolution of the statistics makes you go.

Is my answer useful?

Answer 9

My first response was incorrect -- I was using a result of that postulate to argue for the postulate. Dumb.

From what I remember in my studies, the most rigorous argument is the maximum entropy principle, which states that the system with the highest entropy is the one in which the probabilities of each energy state are equal. The more philosophical arguments include "if you don't know better, assume they are all equal" or "if it doesn't matter, just assume they are all equal".

Now, you might then ask why is entropy being maximized, who says it must? Well, we observe that this seems to be the case in the physical universe. Other than that, you're now entering the realm of metaphysics.

Hope that helps. I know it doesn't!

Answer 10

I think what's happening is that you're temporarily forgetting that energy is conserved.

So the claim is, if I have an isolated system whose volume, numbers of particles of various species, and internal energy are all fixed, then any small range of configurations of particle positions and momentums $X_i \in (x_i, x_i + dx_i), P_i \in (p_i, p_i + dp_i)$ has probability proportional to this phase-space volume $dx_1~dx_2~\dots dx_N~dp_1~dp_2~\dots dp_N.$

Why do we think this claim is true? Because we think that the entire physics is embedded in a Hamiltonian of the overall system, and there is a theorem about how probabilities flow under a Hamiltonian: Liouville's theorem states that the "local" probability distribution must remain constant following the trajectories of the system, while informally we understand that we are uncertain about the state of the system and this uncertainty multiplies as we see interactions within the system. The only good way to reconcile both is to pick out the conserved quantities that are not up for negotiation, look at the hypersurface which has those constant values, and imagine that we're totally uncertain about what point we're on in this hypersurface except that it's uniform in phase space: this satisfies Liouville's theorem, and our idea that our uncertainty about the microscopic state should somehow be maximized, and our desire to say "there's nothing else happening here except for the Hamiltonian dynamics."

Now how do we reconcile this with the "want" of a system to get into a lower energy state? Well, what does "want" mean here? What we mean is maybe that a particle feels a force that goes like the negative gradient of its potential energy, and friction forces usually sap away its kinetic energy, and if we follow both of those we find ourselves in a total energy minimum as the potential becomes kinetic and the kinetic drains away. But we know that this is not the whole story: because we know that (a) friction forces are difficult to model with straight Hamiltonians, and (b) the air we are breathing right now is made out of constantly interacting molecules which do not seem to have decided to all fall down onto the ground, where they would minimize their potential energy.

In fact we can address both by imagining that, as the Hamiltonian approach forces upon us, energy is part of a time-translation symmetry and is globally conserved. So these air molecules cannot "drag" against other things without also "kicking" those things, and they "kick" back, too. One particle's desire to minimize its potential energy is always competing with all the other particles' desire to do the same.

After that the rest of the answer belongs to @knzhou above. We find that whenever we partition one "big isolated system" into two parts, a "test system" and its "environment", that test system will begin to lose energy to that environment -- within reason. So you are moving through this atmosphere on a bicycle, and the kinetic energy of your forward-moving degrees of freedom slowly leaks into these much-lower-energy degrees of freedom of the environment, and this is what you call "air drag." But what's changed is that we know that when you have finally stopped, that actually you have not stopped: your bicycle is constantly being bumped back and forth by the air molecules, and assuming you can stay perfectly balanced on it, you would gradually see a one-dimensional random walk that we refer to as a "Brownian motion" befall you. The only reason that we do not see this much in practice is just that you are very large relative to the size of the atoms; Brown originally observed it for tiny grains of pollen in water under a microscope, and it was the subject of one of Einstein's 1905 "annus mirabilis" papers to connect this random jiggling of grains of pollen to the underlying jiggling of water molecules and argue that it could be used to calculate the size of the atoms themselves.

Answer 11

I think a useful additional way to try and justify or understand the idea of equal a priori probabilities is through the microscopic reversibility of the dynamics of the system. For an isolated system with a fixed energy and composition, we could postulate that the dynamics of the system are reversible; I think intuitively for most people, this seems more obvious than the idea of equal a priori probabilities.

Given that the dynamics of the system are reversible, the transitions to get from any one state to another are equally likely as the transitions which take the system in the other direction. So long as all of the states are accessible (ergodic hypothesis), over a sufficiently long time the system will visit all states with equal probability as there is no preference to transition to one state more often than any other.

Answer 12

Assuming a microcanonical ensemble, 10 particles with 1 J of energy each and 1 particle with 91 J of energy is indeed far more unlikely than 10 particles with 10 J of energy.

Your confusion is that those two sentences define observations, not microstates. Each observation is potentially compatible with multiple microstates, and in most cases the second one comprises many more microstates than the first.