In the microcanonical ensemble, we have the standard Boltzmann expression for entropy: \begin{equation}\label{1} S = k_B\ln \Omega \end{equation} where $\Omega$ is the number of elements of the microcanonical ensemble. It is then stated that this expression is equivalent to the Gibbs expression for entropy $$S = -k_B\sum_{i}p_i\ln{p_i}$$ and maximizing the Gibbs entropy for the canonical ensemble then leads to the Boltzmann distribution. I am struggling to find a rigorous argument for why the Gibbs and Boltzmann expressions are equivalent that does not make use of the Boltzmann distribution (since we then use the Gibbs expression to derive the Boltzmann distribution). The argument given in my class involved computing the entropy of the overall ensemble and then dividing by the number of elements in the ensemble to compute the "entropy of a representative element," but it was never explained why one can do this (i.e. why the entropy of the system is equal to the entropy of the ensemble divided by the number of elements). I was hoping someone could either provide a more rigorous foundation for that argument or provide a sound argument for why the Boltzmann and Gibbs formulas are equivalent. Furthermore, since the Gibbs formula is used outside of the microcanonical ensemble, it stands to reason that the equivalent Boltzmann entropy expression also can be used outside of the microcanonical ensemble. In this case (for example, in a canonical ensemble), how do we interpret $\Omega$?


1 Answer 1


The premise is wrong: they are not always equivalent.

The Gibbs entropy formula gives information entropy for any discrete probability function $p_i$, even for such function that has no use in physics. It is a mathematical concept, a characterization of the discrete probability function, a real positive measure of uncertainty about the microstate. For a probability distribution localized at a single state it gives zero information entropy (zero uncertainty), and the broader the set of states over which the distribution is non-zero, the larger the information entropy is (larger uncertainty about the microstate).

The Boltzmann entropy formula gives statistical-physics-based estimate/determination of thermodynamic (Clausius) entropy of an isolated system with definite volume $V$, number of particles $N$ and energy $U$ (in case of ideal gas; for more complicated systems, entropy may depend also on additional macroscopic state variables, such as magnetic field or surface area). It gives a completely different thing.

The relation between these two different entropy concepts is that the Gibbs formula (giving information entropy of a probability distribution) gives morally the same value as the Boltzmann formula, if we have a very large system with very large number of available microstates, and the probability distribution $p_i$ is such that it maximizes the information entropy under the constraints defined by values of the macroscopic state variables $U,V,N,...$ as used in the Boltzmann entropy.

The microstates $i$ describe the system of $N$ particles in volume $V$, and the probabilities satisfy the constraints

$$ \sum_i p_i E_i = U. $$ $$ \sum_i p_i = 1. $$ The first constraint is that the probability distribution implies average expected energy $U$, and the other is necessary if $p_i$ are to be probabilities, they have to sum up to one.

With these constraints, the distribution that maximizes the Gibbs entropy is the Boltzmann distribution $Ce^{-E_i/kT}$, and value of the Gibbs entropy is the same as that of the Boltzmann entropy for the same macroscopic state:

$$ S_{Gibbs}(\{p_i\}_{constraints~U,V,N,...}) \approx S_{Boltzmann}(U,V,N,...). $$

The approximation sign is there because it takes the limit $N\to\infty$ for the two formulae to behave the same as functions of $U,V,N$, and also because the number of allowed states $\Omega$ can be defined in different ways, giving somewhat different results. These however cause the two entropies to differ only by an additive constant, thus this is immaterial in thermodynamics - the only thing we require of formula for thermodynamic entropy is that it gives change of entropy between two macroscopic states correctly.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.