# How to justify the entropy maximum postulate using Statistical Mechanics?

The entropy maximum postulate states that given a thermodynamic system there's a function $S$ of the extensive parameters called entropy which has the property that once a constraint is removed the equilibrium state the system goes to is the one which maximize $S$ among the available states.

I took a statistical mechanics course recently and in the statistical mechanics approach one considers the number of available microstates $\Omega$ corresponding to some macroscopic state characterized by certain extensive parameters. In that case we have $\Omega = \Omega(X_1,\dots,X_n)$ being $X_i$ the extensive parameters we are considering.

One particular case would be $\Omega = \Omega(E,V,N)$ with $E$ being energy, $V$ the volume and $N$ the number of particles.

In that approach we define the function $S(X_1,\dots,X_n)=k_B \ln \Omega(X_1,\dots,X_n)$. The main idea them is that we consider that on the thermodynamic limit, which is attained when we let the parameters grow with the densities fixed, this function is the thermodynamic entropy.

After that everything works as in thermodynamics and we them apply the entropy maximum postulate.

I've remained with one question, however. This tells how, from statistical mechanics, we get the entropy and so on, so that we can describe the system using thermodynamic theory. On the other hand, nothing justfied the idea of associating this $S$ to the thermodynamics $S$ and applying the entropy maximum postulate.

I always believed statistical mechanics could justify this postulate. In that case: what justifies identifying this entropy with the one found in thermodynamics? And more importantly, how does one justify, using statistical mechanics, the entropy maximum postulate?

What justifies identifying this entropy with the one found in thermodynamics?

Suppose parameter $X_1$ is contrained to some value $x$. The corresponding number of available microstates is $Ω(X_1=x,X_2,\ldots,X_n)$. This part of phase space is a subspace of the whole one, consisting of all microstates without constraint. So $Ω(X_1=x,X_2,\ldots,X_n)<Ω(X_1,X_2,\ldots,X_n)$, hence $S(X_1=x,X_2,\ldots,X_n)<S(X_1,X_2,\ldots,X_n)$, which meets the definition of thermodynamical entropy (even not in the thermodynamic limit).

How does one justify, using statistical mechanics, the entropy maximum postulate?

Statistical mechanics postulates ergodicity: the point representing a system in phase space will eventually explore all the accessible phase-space volume (given the constraints). This is observed experimentally because systems of many particles are chaotic.

When a constraint is released, accessible volume increases dramatically. The probability the system reaches state $(X_1=x,X_2,\ldots,X_n)$ by chance when it wanders in all states $(X_1,X_2,\ldots,X_n)$ is extremely small, so that it will never happen in course of time (imagine how much time you'd have to wait, after a Joule expansion, for all molecules to get back in one of the chambers by chance). That's why entropy of a system never decreases spontaneously in the thermodynamic limit.

• if the system phase space is discrete can you denote all the macrostates as $\mathbf{\Omega}$ (boldfaced Omega)? Which can then be subscripted as $\mathbf{\Omega}_{X_1,\ldots,X_N}$ ?
– Vass
Jun 25, 2018 at 1:10

There is a combinatorial interpretation of entropy in statistical mechanics that associates the equilibrium state with the state that maximizes entropy.

In the canonical ensemble we are dealing with a collection of $$K$$ macroscopic systems all of which have the same volume $$V$$ and number of particles $$N$$ but not necessarily the same energy. In fact, they are allowed to have arbitrary energies as long as their total energy $$E_\text{tot}$$ is fixed. Given $$K$$ and $$E_\text{tot}$$ we can partition the energy on many different ways. Each partitioning is represents by a distribution

$$(k_1,k_2, \cdots k_i, \cdots)$$

where $$k_i$$ is the number of systems in microstate $$i$$ and $$i=1,2,\cdots$$ runs through all possible microstates. If this distribution is given, the probability of microstate $$i$$ is

$$p_i = \frac{k_i}{\sum_i k_i} = \frac{k_i}{K}$$

But distribution $$(k_1,k_2,\cdots)$$ is not unique. In fact, by the postulate of equal a priori probabilities every partitioning of the total energy is equally probable. How can we identify the equilibrium distributions?

Here is how we resolve this indeterminacy: Each distribution is associated with a multiplicity that is given by the multinomial factor

$$\frac{K!}{k_1! k_2!\cdots}$$

This multiplicity counts the number of ways that we can assign microstates to $$K$$ systems so that $$k_1$$ systems are in microstate 1, $$k_2$$ systems in microstate 2 and so on. The most probable distribution is the distribution $$(k_1^*, k_2^*,\cdots)$$ that maximizes this multiplicity. The maximization of the multiplicity is equivalent to the maximization of its logarithm, since the log is a monotonic function of its argument. Using the Stirling approximation the log of the multinomial is

$$\log \frac{K!}{k_1! k_2!\cdots} = - K \sum_i \frac{k_i}{K}\log\frac{k_i}{K} = - K \sum_i p_i \log p_i$$

The summation on the far right is the entropy functional of probability distribution $$p_i$$:

$$\boxed{S[\{p_i\}] \equiv - \sum_i p_i\vphantom{\sum^i} \log p_i}$$

Conclusion The most probable distribution is the distribution that maximizes the multinomial coefficient, equivalently, the distribution that maximizes the entropy functional $$S[\{p_i\}]$$. If $$S^*$$ is the entropy of the most probable distribution and $$S$$ the entropy of any other distribution, then

$$\boxed{~S^* > S \vphantom\int~}$$

This is a statement of the second law.