28
$\begingroup$

As I'm sure folks here know, the principle of equal a priori probabilities, sometimes called the fundamental postulate of statistical mechanics, states the following:

For an isolated system with an exactly known energy and exactly known composition, the system can be found with equal probability in any microstate consistent with that knowledge.

This isn't sitting right in my head because I also know that particles want to occupy the lowest possible energy.

How can it be that all microstates (consistent with the given macrostate - i.e., number of particles N, total volume V, and total energy E) have equal probability, when we know that particles want to occupy lower-energy states? Wouldn't the microstates which have more particles distributed in lower-energy states have a higher probability?

The principle of equal a priori probabilities implies to me that if we had a total energy of E = 100 J and N = 10 particles, it's equally likely to have

  • 10 particles all with 10 J of energy or
  • 9 particles with 1 J of energy each and 1 particle with 91 J of energy.

Whereas, to me, the second one seems intuitively far more unlikely! But each of these microstates is consistent with the overall macrostate. And the macrostate makes no statement regarding the specific energies of each particle. How do we reconcile the principle of equal a priori probabilities with the fact that particles want to assume the lowest possible energy states?

$\endgroup$
5
  • 8
    $\begingroup$ What is this principle according to which particles want to occupy the lowest energy? $\endgroup$ Jan 12 '18 at 20:39
  • 9
    $\begingroup$ Actually, in your example, the second arrangement is 10 times more likely. There is only one microstate corresponding to 10 particles all with 10 J of energy (specifically, all particles with the same energy), while there are 10 microstates corresponding to 9 with 1 J and 1 with 91 J, corresponding to 10 possibilities for the choice of which particle has 91 J. $\endgroup$
    – pwf
    Jan 12 '18 at 21:03
  • $\begingroup$ Related (and possibly a duplicate): How does the fundamental assumption of statistical physics make sense? $\endgroup$ Jan 12 '18 at 21:24
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – ACuriousMind
    Jan 13 '18 at 10:09
  • $\begingroup$ It's worth pointing out that basic statistical mechanics starts by examining non-interacting particles. The reason it's more likely to find the particles of a gas spread around the room and not bunched in the corner is not because of repulsive forces minimizing the interaction potential. It is because there are more ways to spread out than there are to bunch up. You are correct that there are many naturally occurring scenarios where high energy particles gradual leech off their energy to other's nearby, but that necessarily requires interaction. $\endgroup$
    – Geoffrey
    Jan 13 '18 at 17:32

10 Answers 10

17
$\begingroup$

The "search" for lowest energy states is a transient effect. If a system is in a high energy state and can emit energy out of the system, it will do so until the energy of the system diminishes to where it is no longer energetically favorable to do so.

This "isolated system" deals with the steady state case, where energy is not entering or leaving for an arbitrarily long time. Steady state systems exhibit different behaviors than transients.

Given two microstates A and B, we know that the probability of a A->B transition occurring must be the same as the probability of B->A because both states have equal energy. It's easy to see that if the probability of those transitions are equal, then the transient case would have smoothed them out before we arrived at steady state. If P(A) < P(B), we would expect more B->A transitions than A->B transitions, by Bayes' theorem, until eventually we reach P(A) = P(B).

Worth noting however: postulates are not provable. It's called the fundamental postulate of statistical mechanics because we can't actually prove that systems will do this. However, all systems yet observed do indeed demonstrate this behavior, even systems that were designed to break it. You could, however, turn it around and say that if the micro-states are not equiprobable, then you by definition do not fully know the composition of the system.

$\endgroup$
10
  • $\begingroup$ Okay, but in the case where one particle has 91 J and the rest have 1 J each, why doesn't a transfer of energy occur? I mean, nothing is stopping that 91-J particle from emitting some of that energy (e.g., in the form of a photon) and undergoing a relaxation to a lower energy level, with another particle then absorbing the photon. $\endgroup$
    – Martensite
    Jan 12 '18 at 22:35
  • $\begingroup$ It seems to me that a high-energy particle is more likely to release some energy than a low-energy particle, meaning that high-energy particles will keep undergoing relaxation until every particle has the same probability to release/absorb energy (i.e., when they are all equal at 10 J). $\endgroup$
    – Martensite
    Jan 12 '18 at 22:36
  • 9
    $\begingroup$ The trick, of course, is to define the microstates such that their volumes are equal. This means that the "postulate" is really more of a design constraint; it's best to select our descriptions of microstates to be consistent with it, rather than assume that any random description we happen to pick is already consistent. $\endgroup$
    – Nat
    Jan 12 '18 at 23:17
  • 2
    $\begingroup$ @Martensite, yes, the "exactly even" state is even less likely than the "very uneven" state, but that's reasonable, why would the energy be likely to be exactly even? Instead, compare the number of ways to get "one particle has energy 21, nineteen particles have energy 1" (very uneven, only twenty combinations) with the number of ways to get "ten particles have energy 1, ten particles have energy 2" (pretty even, and, uh, lots of combinations). $\endgroup$ Jan 12 '18 at 23:37
  • 1
    $\begingroup$ @Nat This! Or else we end up thinking that there is a 50% chance to win the powerball lottery: Either you win, or you lose, both iwth equal probability ... $\endgroup$ Jan 14 '18 at 21:52
7
$\begingroup$

The postulate of equal probabilities for each microstate is perfectly compatible with the observation that macroscopic objects try to minimize their energy.

As a concrete example, suppose I toss $2N$ coins, and I know that $N$ of them come up heads. We think of the heads as representing energy. Now split the coins into two equal piles, the first $N$ I tossed and the last $N$. How are the heads/"energy" typically distributed between these piles?

For $N = 1$, the configurations are $(H, T)$ and $(T, H)$, so one pile has all of the energy with probability $1/2$. For $N = 2$, the configurations are $$(HH, TT), (HT, HT), (HT, TH), (TH, HT), (TH, TH), (TT, HH)$$ which is already dramatically different: there's a $4/6$ chance for a perfectly even split of energy, and only a $1/6$ chance for each pile to have all the energy.

This trend continues and intensifies for higher $N$. When $N = 100$, there's still only one configuration where the first pile has no energy, but $10000$ ways for the first pile to have one unit of energy. And there are over $10^{50}$ ways to have an equal split of energy. If you do the math, you can show that it's very likely that the split is pretty close, within $25\%$. And in the real world, $N \sim 10^{23}$!


To put some more formalism around this, the number of ways a pile of coins can have a given amount of energy is measured by the entropy, with the total entropy of the two piles maximized when the energy is evenly split. And while every configuration is still equally likely, and it's completely possible to have a totally uneven split of energy, it is overwhelmingly more likely to have a nearly even split.

When you imagine a hot object cooling off, imagine a small pile of coins that's mostly heads, inside an enormous sea of coins that's mostly tails. If you start randomly swapping coins around, you expect the heads to disperse with overwhelming probability, even if every swap is equally likely.

There's a nice way to characterize when this process ends. For each subsystem, we compute how much energy it needs per unit change in entropy, $$T = \frac{dU}{dS}.$$ Then the entropy is maximum once each subsystem reaches the same "marginal entropy rate", i.e. $T_1 = T_2$. But our $T$ is actually the definition of temperature. We've just shown that heat flows from hot to cold, though the underlying random coin flips couldn't care less.

$\endgroup$
6
  • $\begingroup$ You said "The postulate of equal probabilities for each microstate is perfectly compatible with the observation that macroscopic objects try to minimize their energy." but this confuses me, because microscopic objects try to minimize their energy too. The individual particles will surely also be trying to attain the lowest energy level, no? $\endgroup$
    – Martensite
    Jan 12 '18 at 22:37
  • $\begingroup$ @Martensite If they're interacting with a very large system, absolutely, by the same coffee cup analogy I used above. I should have said "objects in a macroscopic environment". $\endgroup$
    – knzhou
    Jan 12 '18 at 22:41
  • $\begingroup$ I'm really struggling with this and I don't know why. Can you help me understand why, given two options "1 particle with 9 J and 1 particle with 1 J" and "2 particles with 5 J each", the second is not more likely? I mean, the first system has the very high-energy 9-J particle and it seems to me like it would transfer some of that energy to the 1-J particle. $\endgroup$
    – Martensite
    Jan 12 '18 at 22:43
  • 1
    $\begingroup$ @Martensite When you're down to the scale of individual particles, everything is completely chaotic. Don't imagine one particle walking up to the other and just gradually handing over some of its energy, because particles can't do that. Imagine them rapidly bouncing around in a box and occasionally colliding with each other, with the collision just about equally likely to move energy either way. $\endgroup$
    – knzhou
    Jan 12 '18 at 22:46
  • 2
    $\begingroup$ @Martensite One analogue of this is to look at a simulation of a double pendulum; there are two particles with fixed total energy but no reason to expect that it should be evenly distributed. Occasionally one of them stops while the other picks up all the energy, and the next moment it's reversed. $\endgroup$
    – knzhou
    Jan 12 '18 at 22:47
5
$\begingroup$

It's a design constraint rather than a fundamental truth

It's not inherently "true" so much as a design constraint when constructing descriptions of microstates.

For example, consider a description of microstates in which it is true that all of them are equally probable. Then, redefine the ensemble such that it's exactly the same, except half of the original microstates are now described as a single microstate that encompasses all of them. Obviously, they're not equally probable anymore.

So,

For an isolated system with an exactly known energy and exactly known composition, the system can be found with equal probability in any microstate consistent with that knowledge.

can be restated as:

When you're constructing a description for your ensemble of microstates, try to define them such that they're all equally likely.

Such physical models don't tend to require perfection, so it's not like everything'll instantly break if there's a small error in the description of microstates where some are a bit more probable than others.

I mean, sure, we try to fit our models to reality as best-as-possible, but some error's gonna happen.

For example, folks often talk about ensembles of coins that can be Heads-or-Tails. The obvious microstate description for a bunch of such coins is where each microstate corresponds to one set of Heads-or-Tails value for each coin. But even if we select that description, which sounds fair enough, coins don't tend to be perfectly fair, so the description won't be perfect.

A real-life scientific example of that is with chemical isotopes. This is, it's possible to ignore the differences between isotopes of the same element; this introduces some error, but in common practice it's generally allowed as a reasonable approximation unless there's some compelling reason not to.

Basically, the trick's to select microstates that're approximately equally likely. The better the fit, the more logically consistent the arguments based on it would tend to be.

Entropy tends to grow due to degeneracy

Statistical Mechanics is all about how degenerate states are more likely.

For example, an ideal gas is said to evenly disperse about a room because there're many more microstates in which it is rather than, say, all of the gas particles being smooshed together in some corner of the room.

This isn't that any single dispersed microstate is more likely, but rather there're way more dispersed microstates than non-dispersed microstates. This is:

  1. Each dispersed microstate should still be as likely as any other microstate.

  2. However, there're astronomically more dispersed microstates, such that they're collectively more likely than the collection of their alternatives.

However, say that you redefine the set of all states in which gas particles are dispersed about the room to be a single microstate. Then, yes, that one micro-state would be far more likely than any other. The fundamental postulate of Statistical Mechanics simply recommends that you don't define them that way, since the heterogeneity of it screws with other descriptions.

$\endgroup$
14
  • $\begingroup$ It's not really a design constraint. Microstates are by definition just quantum states, like the orbits an electron can have around a proton. You can't just pick and choose those as you want; you can't group two energy levels of an atom together and say they're the same. $\endgroup$
    – knzhou
    Jan 13 '18 at 8:58
  • 2
    $\begingroup$ @knzhou That's not correct, nor am I sure why someone would think that. Statistical Mechanics predates Quantum Mechanics, and it applies more generally to other informational systems; even if we happen to select quantum states by convention, that can't be the definition as it'd only be applicable in a small sub-domain. $\endgroup$
    – Nat
    Jan 13 '18 at 10:19
  • $\begingroup$ The distribution is also fixed in classical mechanics; it is supposed to be just "uniform density on phase space", where phase space is where Hamiltonian mechanics is set. And you're not allowed to arbitrarily stretch and shrink phase space because that messes up Hamilton's equations. $\endgroup$
    – knzhou
    Jan 13 '18 at 10:44
  • $\begingroup$ The point is, maybe if you use an analogue of statistical mechanics in computer science the microstates are up to you to design, but in both cases it's used in physics the definitions are fixed. There's really physical content here, it's not true by fiat. $\endgroup$
    – knzhou
    Jan 13 '18 at 10:45
  • $\begingroup$ @knzhou The arbitrariness remains in classical mechanics (Wikipedia), such that microstates remain constructed. Quantum states do appear to have meaningful physical distinctions, and to a rough approximation it can be convenient to associate them with statistical states, but the two really are fundamentally different concepts. $\endgroup$
    – Nat
    Jan 13 '18 at 10:51
2
$\begingroup$

The principle of equal a priori probabilities implies to me that if we had a total energy of E = 100 J and N = 10 particles, it's equally likely to have

  • 10 particles all with 10 J of energy or
  • 9 particles with 1 J of energy each and 1 particle with 91 J of energy.

Whereas, to me, the second one seems intuitively far more unlikely!

If the particles are identical, there is only one possible microstate corresponding to each one of these two cases, so you have a priori no reason to say that one state is more unlikely.

Intuitively, I believe that you think that the second case is less likely because there is one particle with energy far higher than all the others, and indeed there are not many microsates in which a single particles has energy far higher then the others (try to write down the possible states in which a single particle has energy greater than, say, $50$ and you will see what I mean). However, this is not in contrast with the statement that all the microstates are equally probable!

Be sure that you are not confusing the probability that a single particle has energy greater than $x$ (which is calculated over all the microstates) with the probability of a single microstate in which one of the particles happens to have energy greater than $x$...

$\endgroup$
2
$\begingroup$

I think the fundamental source of your confusion is related to the following idea from statistics:

Imagine I have a coin that I flip 10 times. Intuitively, a series of flips that came out as $HHTHTTHTTH$ would be "expected", while the series $HHHHHHTHHH$ would be "unexpected". If we think about the problem carefully, however, we can see that the exact state $HHTHTTHTTH$ is precisely as unlikely as the state $HHHHHHTHHH$; they both have probability $0.5^{10}$.

The reason for the confusion is that our human brains love seeking patterns, so instead of looking at exact microstates, we tend to think about the "big picture", even when it leads us astray. So for instance, $HHTHTTHTTH$ seems more likely because there are many more states with 5 heads than there are with 9 heads. Our brain is so used to only dealing with the end results of probability that we forget that individual outcomes are not the same as the aggregate events that we care about-- mainly because there are so many outcomes of most systems that it's basically impossible to look at them individually. Time for a relevant Feynman quote:

"You know, the most amazing thing happened to me tonight... I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!"

Similarly with your example, in real life we're normally not actually concerned with exact micro states, given that each one by itself is so unlikely. So, many people intuitively consider a state where all particles have very small fluctuations away from 10 J to be the same as one where they all have exactly 10 J, even though they don't consciously acknowledge this assumption. But a state where one particle has almost all the energy is less stable with respect to small fluctuations, since the fluctuations of the many particles with little energy can add up to take a significant amount away from the particle with lots of energy.

TLDR: I think your problem is that you're unconsciously conflating individual outcomes with aggregate events.

$\endgroup$
2
  • $\begingroup$ Great answer. This is what I would answer if you hadn't done it already. Long story short, his intuition is incorrect. And this mistake is very common in statistics, as you pointed out. $\endgroup$
    – Pedro A
    Jan 14 '18 at 1:23
  • $\begingroup$ This is a very interesting point: we intuitively think of configurations which are representative of highly likely events as being highly likely themselves. But in fact, "entropically likely" events get that way by simply having lots of distinct ways to occur, each equally likely or close to it. I guess the shorthand "maxim" version of this is "it is human instinct to use coarse grained descriptions". $\endgroup$
    – Ian
    Jan 14 '18 at 18:45
1
$\begingroup$

You can think of this as assigning or adding energy to a random particle in the system, 1 J at a time (pretending energy is discretized at 1 J intervals, to simplify).

You have 100 J that you need to assign to 10 particles. Who are you going to give the first one to? You have no reason to prefer any at this point, so you give it to a random one.

Now you have 99 J. Who are you going to give your second one to? Here you need to consider one of two reasonable assumptions:

  1. There is still no reason to prefer any particle. You continue to assign each unit of energy randomly. This is like rolling a 10-sided die, 100 times. Without doing any math, you expect each number to come up roughly 10 times. Likewise, you expect each of your particles to end up with roughly 10 J of energy. OR
  2. Since the particles now have different energies, you may develop a preference. But then, do you prefer to assign your energy unit to particles with low energy or to particles with high energy? You can design some thought experiments and potentially argue for either case, depending on the system you come up, how you imagine the particles, how you add the energy...

A good system to consider is this: you have lots (billions) of particles in a small box, and you add heat to the system by putting one side of the box in contact with an external heat source. Thus, any particle in your box-system that comes in contact with the "hot" side of the box gains a unit of energy before bouncing off. If the particles don't interact with each other (they move in straight lines), then once a particle hits the hot side, it will take a relatively long time for it to reach the other side of the box and come back to hit the hot side a second time (even if its velocity increased slightly due to the energy it gained). Thus, it is likely that every other particle will hit the hot side once before this particle hits it a second time.

Of course, you can and should try to come up with your own systems to test your intuition.

$\endgroup$
1
$\begingroup$

$91J$ is less likely than $10J$, but $10J$ is less likely than $1J$.

Judging from the comments, you are making a critical oversight in your approach to the problem.

You are focusing on the fact it is extremely improbable that one particle would have 91J of energy rather than 10J of energy.

What you are overlooking is that it is also extremely improbable that nine particles would have 10J of energy rather than 1J of energy.

If we carry this heuristic argument through by plugging the numbers into the probability density function $\exp(-E / (kT))$, the two heuristic probability densities are:

$$\left( \exp(-10 / (kT)) \right)^{10} = \exp(-100 / (kT)) $$ $$\left( \exp(-1 / (kT)) \right)^{9} \exp(-91 / (kT)) = \exp(-100 / (kT)) $$

which are exactly the same.

(I assume in the above that you single out a specific particle to be the one with high energy)

$\endgroup$
2
  • $\begingroup$ You are using the canonical probability density, which is derived from the microcanonical one, which is based on the postulate of equal a priori probability... $\endgroup$
    – valerio
    Jan 13 '18 at 10:52
  • $\begingroup$ @valerio92: Yes, but so was the OP (check the chat link in the comments to the main question). This answer is intended to address the OP's question and concerns as they are explained in his post and comments, rather than to ignore those and respond to the title in isolation. $\endgroup$
    – user5174
    Jan 13 '18 at 11:40
1
$\begingroup$

Imagine that the system has its interactions initially turned off. In that case, why are the particles going more likely to be in one state or the other, if the state is compatible with your knowledge? If the particles interact, you can use a state that is compatible with the probability distribution of the system evolving from one of the equally likely states to where the time evolution of the statistics makes you go.

Is my answer useful?

$\endgroup$
0
$\begingroup$

My first response was incorrect -- I was using a result of that postulate to argue for the postulate. Dumb.

From what I remember in my studies, the most rigorous argument is the maximum entropy principle, which states that the system with the highest entropy is the one in which the probabilities of each energy state are equal. The more philosophical arguments include "if you don't know better, assume they are all equal" or "if it doesn't matter, just assume they are all equal".

Now, you might then ask why is entropy being maximized, who says it must? Well, we observe that this seems to be the case in the physical universe. Other than that, you're now entering the realm of metaphysics.

Hope that helps. I know it doesn't!

$\endgroup$
4
  • 1
    $\begingroup$ The law of the entropy increase is not just an experimental fact. It also is theoretically derived in statistical mechanics. $\endgroup$
    – safesphere
    Jan 12 '18 at 21:24
  • $\begingroup$ That being the same statistical mechanics for which the fundamental postulate is in question, right? @safesphere $\endgroup$
    – Nij
    Jan 13 '18 at 1:52
  • $\begingroup$ @Nij I will leave the details to the experts. I was merely pointing out the fact that law of the entropy increase had theoretical grounds in addition to observations. $\endgroup$
    – safesphere
    Jan 13 '18 at 3:46
  • $\begingroup$ The question asks about whether the fundamental postulate is valid. Statistical mechanics is based on that postulate. Answering that "it must be valid because we can derive a theoretical law based on... that postulate..." $\endgroup$
    – Nij
    Jan 13 '18 at 3:55
0
$\begingroup$

I think what's happening is that you're temporarily forgetting that energy is conserved.

So the claim is, if I have an isolated system whose volume, numbers of particles of various species, and internal energy are all fixed, then any small range of configurations of particle positions and momentums $X_i \in (x_i, x_i + dx_i), P_i \in (p_i, p_i + dp_i)$ has probability proportional to this phase-space volume $dx_1~dx_2~\dots dx_N~dp_1~dp_2~\dots dp_N.$

Why do we think this claim is true? Because we think that the entire physics is embedded in a Hamiltonian of the overall system, and there is a theorem about how probabilities flow under a Hamiltonian: Liouville's theorem states that the "local" probability distribution must remain constant following the trajectories of the system, while informally we understand that we are uncertain about the state of the system and this uncertainty multiplies as we see interactions within the system. The only good way to reconcile both is to pick out the conserved quantities that are not up for negotiation, look at the hypersurface which has those constant values, and imagine that we're totally uncertain about what point we're on in this hypersurface except that it's uniform in phase space: this satisfies Liouville's theorem, and our idea that our uncertainty about the microscopic state should somehow be maximized, and our desire to say "there's nothing else happening here except for the Hamiltonian dynamics."

Now how do we reconcile this with the "want" of a system to get into a lower energy state? Well, what does "want" mean here? What we mean is maybe that a particle feels a force that goes like the negative gradient of its potential energy, and friction forces usually sap away its kinetic energy, and if we follow both of those we find ourselves in a total energy minimum as the potential becomes kinetic and the kinetic drains away. But we know that this is not the whole story: because we know that (a) friction forces are difficult to model with straight Hamiltonians, and (b) the air we are breathing right now is made out of constantly interacting molecules which do not seem to have decided to all fall down onto the ground, where they would minimize their potential energy.

In fact we can address both by imagining that, as the Hamiltonian approach forces upon us, energy is part of a time-translation symmetry and is globally conserved. So these air molecules cannot "drag" against other things without also "kicking" those things, and they "kick" back, too. One particle's desire to minimize its potential energy is always competing with all the other particles' desire to do the same.

After that the rest of the answer belongs to @knzhou above. We find that whenever we partition one "big isolated system" into two parts, a "test system" and its "environment", that test system will begin to lose energy to that environment -- within reason. So you are moving through this atmosphere on a bicycle, and the kinetic energy of your forward-moving degrees of freedom slowly leaks into these much-lower-energy degrees of freedom of the environment, and this is what you call "air drag." But what's changed is that we know that when you have finally stopped, that actually you have not stopped: your bicycle is constantly being bumped back and forth by the air molecules, and assuming you can stay perfectly balanced on it, you would gradually see a one-dimensional random walk that we refer to as a "Brownian motion" befall you. The only reason that we do not see this much in practice is just that you are very large relative to the size of the atoms; Brown originally observed it for tiny grains of pollen in water under a microscope, and it was the subject of one of Einstein's 1905 "annus mirabilis" papers to connect this random jiggling of grains of pollen to the underlying jiggling of water molecules and argue that it could be used to calculate the size of the atoms themselves.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.