# Deriving the canonical ensemble from the microcanonical ensemble: Why expand the logarithm of the probability and not some other function

Several other posts on the derivation of the canonical ensemble give an explanation in terms of considering a subsystem of a larger microcanonical ensemble see How is the distribution probability in the canonical ensemble derived? or the second response in (Canonical) Partition function - what assumption is at work here?.

But these derivations involve a step where one has that the probability of finding the subsystem with an energy $$E_s$$ is given by the probability of finding the bath/reservoir with an energy $$E_b=E_{tot}-E_s$$, i.e.

$$p_s(E_s)=\frac{\Omega_b(E_b)}{\Omega_{tot}}$$

where $$\Omega$$ denotes the number of microstates.

They then say that $$E_s< and so we can expand $$\ln(\Omega_b(E_b))$$ to first order around $$E_{tot}$$ i.e.

$$\ln(\Omega_b(E_b))=\ln(\Omega_b(E_{tot}))-E_s\frac{\partial\ln(\Omega_b)}{\partial E_{tot}}$$

But this seems like it is just assuming the canonical distribution. Why should we choose to expand $$\ln(\Omega_b)$$ to first order and not some other function of $$\Omega_b$$?

The applicability of the canonical ensemble depends on the form of $$\Omega(E)$$. For some $$\Omega(E)$$, the canonical ensemble can't even be defined, much less derived! The canonical ensemble applies only in systems for which the quantity $$\frac{\partial \log\Omega(E)}{\partial E}$$ is a decreasing function of $$E$$. The point is that the derivation of the canonical ensemble from the microcanonical ensemble relies on an assumption about the form of the function $$\Omega(E)$$, as well as on a thermodynamic limit. These two things are the reason why we keep the first-order term in the expansion of $$\log\Omega(E)$$ instead of in the expansion of some other function of $$\Omega(E)$$. This is illustrated below with two explicit examples.

There are systems of interest that do not satisfy these conditions, and for those systems, the canonical ensemble is not applicable, at least not strictly (though it may still be an excellent approximation). So, in a sense, the textbook derivation of the canonical ensemble from the microcanonical ensemble does assume the canonical ensemble! More accurately, the derivation assumes certain conditions that are true of many systems of interest, and those are the conditions under which the canonical ensemble is applicable. The following examples illustrate those conditions.

## Example 1: Ideal gas

The entropy of an ideal gas is $$S(E)=N\log V +\frac{ND}{2}\log E \tag{1}$$ where $$N$$ is the number of atoms, $$V$$ the total volume, $$E$$ the total energy, and $$D$$ the number of spatial dimensions (normally $$D=3$$). The number of states is $$\Omega(E)\propto e^{S(E)}. \tag{2}$$ If we partition the system into two parts, a large part $$L$$ and a small part $$S$$, then assuming the microcanonical ensemble for $$L+S$$ with total energy $$E$$ is equivalent to assigning the probability $$p(\epsilon)\propto \Omega(E-\epsilon) \tag{3}$$ to each state (each energy eigenstate in the quantum case) of $$S$$ with energy $$\epsilon$$. Now, consider the expansion $$S(E-\epsilon)=S(E)-\epsilon\frac{\partial S}{\partial E} +\frac{\epsilon^2}{2}\frac{\partial^2 S}{\partial E^2} +\cdots \tag{4}$$ Equation (1) implies $$\frac{\partial^n S}{\partial E^n} \propto\frac{N}{E^n} \tag{5}$$ with a coefficient that is independent of $$N$$ and $$E$$. This can also be written $$\frac{\partial^n S}{\partial E^n} \propto\frac{1}{N^{n-1}(E/N)^n}. \tag{6}$$ The thermodynamic limit is $$N\to \infty$$ with $$E/N$$ and $$V/N$$ held fixed. Here's the key: The only $$\epsilon$$-dependent term in (4) that survives this limit is the $$n=1$$ term, so equation (3) becomes exactly $$p(\epsilon)\propto \exp(-\beta \epsilon) \hskip2cm \beta := \frac{\partial S}{\partial E}\propto \frac{1}{E/N} \tag{7}$$ in this limit. This is why we expand $$S(E)=\log\Omega(E)$$ instead of some other function of $$\Omega(E)$$, at least in the case of an ideal gas.

## Example 2: Photon gas

Now suppose $$S(E)=\left( \left(\frac{E}{\hbar c}\right)^D V\right)^{1/(D+1)}. \tag{8}$$ This is the entropy of a gas of photons. The point of considering this example is that the canonical ensemble still applies even though this system does not involve any given number of particles $$N$$. To define the thermodynamic limit, we can use $$V\to \infty$$ with $$E/V$$ fixed. Equations (2)-(4) still apply here, and equation (5) is replaced with $$\frac{\partial^n S}{\partial E^n} \propto \frac{\big( E^D V\big)^{1/(D+1)}}{E^n} \propto V^{1-n} (E/V)^{D/(D+1)-n} \tag{9}$$ with a coefficient that is independent of $$E$$ and $$V$$. Once again, the only $$\epsilon$$-dependent term in (4) that survives this limit is the $$n=1$$ term, so equation (3) becomes exactly $$p(\epsilon)\propto \exp(-\beta \epsilon) \hskip2cm \beta := \frac{\partial S}{\partial E}\propto (V/E)^{1/(D+1)}. \tag{10}$$ This is why we expand $$S(E)=\log\Omega(E)$$ instead of some other function of $$\Omega(E)$$, at least in the case of a photon gas.

## Summary

The point is that the canonical ensemble is strictly applicable only in a thermodynamic limit and only for a certain class of functions $$\Omega(E)$$. Many systems of interest do satisfy those conditions, and that's the justification for keeping only the first-order term in the expansion of $$\log\Omega(E-\epsilon)$$ rather than in the expansion of some other function of $$\Omega(E -\epsilon)$$.

• Wow, interesting! I never even noticed this. Do you have any examples of systems on hand that don't satisfy these conditions? Jul 18 '19 at 20:20
• @knzhou My favorite example is the astrophysical one: Since the counting of states in a region of radius $R$ is dominated by the entropy of a single big black hole when the energy is high enough, this gives $S(E)\propto \text{horizon area}\propto R^{d-2}\propto E^{(d-2)/(d-3)}$ in $d$ spacetime dimensions, using the black hole mass $M$ as the "energy" $E$ (page 3 in arxiv.org/abs/hep-th/0502050 ). Even at less extreme energies, "gravitating systems of interest in astrophysics cannot be described by a canonical ensemble" (page 2 in arxiv.org/abs/0812.2610 ). Jul 19 '19 at 3:02
• @knzhou Regarding non-astrophysical examples, see arxiv.org/abs/0907.0323 ("Statistical mechanics and dynamics of solvable models with long-range interactions"). Page 5 in arxiv.org/abs/cond-mat/9812172 ("Negative Specific Heat in Astronomy, Physics and Chemistry") says: "two negative specific heat systems can not be in thermal equilibrium, so an equilibrium canonical ensemble of them is impossible." Jul 19 '19 at 3:02

This is explained well in some statistical mechanics texts, I think I first read it in Statistical Physics by F Mandl. But it may be glossed over in some books. The essential point is that the successive terms in such an expansion should get smaller in magnitude, otherwise the expansion will not converge. $$\Omega(E)$$ itself increases incredibly quickly with $$E$$. The logarithm of $$\Omega$$ is essentially the entropy, a thermodynamic extensive quantity, and so its derivative with respect to $$E$$ will be an intensive quantity, well behaved; in fact it ends up being identified with $$1/T$$. So, if $$N$$ is the system size, it is a quantity of order $$1/N$$ in magnitude relative to $$S$$. it doesn't take much further thought to see that the next derivative will be another factor $$1/N$$ smaller, roughly speaking, and so on. If we tried expanding $$\Omega$$ instead, we wouldn't get such a well behaved converging series.

I've slightly glossed over one point in the above. To compare the "magnitudes" of successive terms, I should make sure they have the same units: ideally I should make them dimensionless. This means including the appropriate power of $$\delta E$$, the fluctuation of the energy from the reference energy used for the expansion. We generally assume that $$\delta E$$ is small compared to $$E$$, i.e. not as "large" as an extensive quantity, which is consistent with what I said in the previous paragraph.

• This still seems to essentially be assuming the final result. Why shouldn't you for example expand $\ln(\ln(\Omega))$?
– J.L.
Apr 11 '19 at 15:21
• I think the essential assumption is the extensivity of thermodynamic properties, which implies $S\sim cE$, and hence $\Omega\sim e^{cE}$ in units where $k_B=1$, where $c$ is some constant (for a given state point) and I'm just considering the leading, or most important, term. If you like, it's a physical assumption, but it can be justified if you have a system where, in the limit of large system size, dividing it in half results in two systems whose extensive properties are all halved, i.e. the terms which have been removed by cutting the system in half are relatively insignificant.
– user197851
Apr 12 '19 at 6:46

Because we know that $$\partial_E \ln(\Omega(E)) = \frac{1}{k_B T}$$. Thus taking the logarithm allows to use observables in the equation instead of some crypticaly defined terms.

• How do we know that?
– J.L.
Apr 11 '19 at 16:15
• It's the definition of Temperature... Apr 12 '19 at 5:56