Probability in statistical mechanics

Question

Suppose we've an isolated box having $N$ classical distinguishable particles in it, the box being hypothetically divided into two parts, left and right with both parts identical.

Reif says

...Note that there is only one way of distributing the $N$ molecules so that all $N$ of them are in the left half of the box. It represents only one special configuration of the molecules compared to the $2^{N}$ possible configurations of these molecules. Hence we would expect that, among a very large number of frames of the film, on the average only one out of every $2^{\mathrm{N}}$ frames would show all the molecules to be in the left half. If $P_{N}$ denotes the fraction of frames showing all the $N$ molecules located in the left half of the box, i.e., if $P_{N}$ denotes the relative frequency, or probability, of finding all the $N$ molecules in the left half, then $$ P_{N}=\frac{1}{2^N} . $$

Why should we expect that, among a very large number of frames of the film, on the average only one out of every $2^{\mathrm{N}}$ frames would show all the molecules to be in the left half?

I have the intuition for it but how can one show this should be the case indeed rigorously?

Why can't the average be more or less than the above value? Also how large is "very large frames of film" here?

Answer 1

We assume that all the configurations are equally probable - this is one of the principal assumptions of statistical physics. It is like throwing a die with $2^N$ sides - you expect each side appear on average once in $2^N$ throws.

Update
As discussed in the comments, the argument above is understandable intuitively, but hard to make rigorously. It is essentially a frequentist argument, assuming that we can have an infinite number of trials (or observe the system for a very long time, if we assume ergodicity). The alternative, Bayesian probability, openly acknowledges that the equiprobability of the events is a matter of our belief, based on our previous experience, but not necessarily justified in any rigorous sense. Eventually, both approaches boil down to belief and intuition - depending on what kind of these you are willing to accept. Axiomatic foundations of probability do not really give a rigorous answer: they might help developing the mathematical apparatus, but still require postulating the probability distribution.

Relevant quote:
The following quote is attributed to Arnold Sommerfeld:

Thermodynamics is a funny subject. The first time you go through it, you don’t understand it at all. The second time you go through it, you think you understand it, except for one or two points. The third time you go through it, you know you don’t understand it, but by that time you are so used to the subject, it doesn’t bother you anymore.

Remark:
What makes the things even more confusing is that further in studying of statistical physics one encounters the situations where the equiprobability of events does break: everything that can be associated with terms critical phenomena, phase transitions, ergodicity breaking, spontaneous symmetry breaking.

Answer 2

If we have $N$ independent particles each of them has a 50-50% chance to be in the left compartment of the box. Hence, it is analog to throwing a coin and determining for each particle in which compartment it is put -- head corresponds to the left compartment and tails to the right. Now ask yourself, what is the probability that

1. if we have $N=1$ particle that it goes into the left compartment?
2. if we have $N=2$ particles that both are in the left compartment?
3. if we have $N=3$ particles that both are in the left compartment?

Do you see how the $1/2^N$ is the general formula for the probability?

Answer 3

Let $p$ denote the probability of a specific observation type - in this case, $p=2^{-N}$. But never mind the exact value of $p$: the question is why $p$ is a limiting proportion in the large-sample case.

We assume each observation is independent of others in this probability. In $k$ observations, the number that "fit the bill" is $\operatorname{B}(k,\,p)$-distributed, with mean $kp$ and standard deviation $\sqrt{kp(1-p)}$. Equivalently, the proportion has mean $\mu=p$ and standard deviation $\sigma=\sqrt{p(1-p)/k}$, so $\frac{\sigma}{\mu}=\sqrt{\frac{1-p}{kp}}$, which $\to0$ as $k\to\infty$.