Why does $S = k_B \ln W$ not always apply?

Question

I thought for a long time that the Boltzmann formula for entropy, $S = k_B \ln W$, was a universally true statement, or rather the definition of entropy from the perspective of statistical mechanics. However, I have since come to understand that it is only applicable for an isolated system (i.e. the microcanonical ensemble) for which all microstates of the system are equally likely. The more general statement is the Gibbs entropy

$$S = -k_B \sum_i P_i \ln P_i \,.$$

However, I have seen a derivation of the Boltzmann formula such that I can't quite see why said formula doesn't always apply. I was hoping somebody could point out the error in the following reasoning.

From classical thermodynamics we know that

$$T = \left(\frac{\partial U}{\partial S}\right)_V \,.$$

Now let us consider two systems in thermal contact, allowed to exchange energy. If we suppose that all microstates of the joint system are equally likely, we argue that equilibrium will be attained for a division of energy that maximises the number of possible corresponding microstates. So we have

$$ \frac{d}{dE_1}\big(W_1(E_1)W_2(E_2)\big) = 0 \,. $$

Working through this, we get to the condition that

$$ \frac{d \, \ln W_1(E_1)}{E_1} = \frac{d\, \ln W_2(E_2)}{E_2} \,,$$

but since equilibrium corresponds to equal temperatures, we make the natural definition that

$$\frac{1}{k_B T} = \frac{d \, \ln W(E)}{d E}\,,$$ where we choose this particular form of the expression on the left since it results in energy flowing in the right direction (hot to cold) for two systems with very similar temperatures.

Putting this together with the above result, identifying $U =E$, we must have that

$$ S = k_B \ln W \,.$$

Question: where in this argument have I made any assumptions or mistakes, such that this formula applies only to a specific class of systems? Why can I not use this formula to determine the entropy of (say) one of the two systems I placed in thermal contact (in the discussion above of temperature)? Why is the Gibbs formula the correct one for systems allowed to exchange energy? We also use, I believe, this definition of temperature in the derivation of the canonical/Boltzmann distribution (see e.g. here), and yet in this case the reservoir is not an isolated system, and so I would have thought that this expression would not apply. Thank you.

Answer 1

The problem with the Boltzmann definition is, as you have neatly shown, that its usefulness depends on the assumption that your system is in equilibrium with its surroundings. Without first assuming equilibrium and subsequently setting the temperatures as equal, one cannot show that the Boltzmann entropy satisfies the First Law and hence meaningfully define it as the entropy. However the Gibbs entropy still does provide a meaningful definition because, for example, it is possible to relate it to the partition function via

$$S \equiv -k_B\sum_s P_s\ln P_s= k_B \left(\ln Z + \beta\frac{\partial \ln Z}{\partial \beta}\right)=\frac{\partial}{\partial T}(k_B T \ln Z)$$

and hence to use it to calculate other variables like the Helmholtz free energy.

Furthermore, you are assuming that $W$ is large enough to be approximated as a continuous quantity. $W$ is however an integer and the quantity $\mathrm d\ln W$ is not well defined in a system with a small number of microstates. Hence the justification you have provided here would break down. This problem never arises if you start from the definition of the Gibbs entropy and work from there.

Answer 2

Question: where in this argument have I made any assumptions or mistakes, such that this formula applies only to a specific class of systems?

The only assumption made is that the two systems connected interact weakly, so that when the first system has average energy around $E_1$, the macrostate of the joint system has phase volume $W_1(E_1)W(E-E_1)$. I think this is justified for all systems of classical thermodynamics. If the systems interacted strongly and the energy was not homogeneous function of number of particles and volume, the phase volume may not have been given by such formula.

Why can I not use this formula to determine the entropy of (say) one of the two systems I placed in thermal contact (in the discussion above of temperature)?

You can; even if the sub-system 1 interacts with 2 and its energy $E_1$ varies in principle, for macroscopic systems this variation is negligible and only the average value $U_1 = \langle E_1\rangle$ is important for the entropy. One can treat the sub-system as if it was isolated system of the same volume and energy given by $E_1 = U_1$.

Why is the Gibbs formula the correct one for systems allowed to exchange energy?

The Gibbs formula is not "the correct" formula for entropy. Rather the point of the Gibbs formula is that it is a functional of the probability distribution $p_k$, with the important property that the maximum possible value of this functional for system with prescribed average energy $U_1$ gives entropy at this average energy; numerically, $k_B \ln W_1(E_1)$ and $\max_{\sum_k \epsilon_k p_k = U_1}\left\{ k_B\sum_k -p_k\ln p_k\right\}$ are the same for dense systems of countable states $k$ ($\epsilon_k$ is energy of the state $k$.)

Answer 3

The entropy defined by Boltzmann can be applied in systems with fluctuating energy. The only requirement is the sharp definition of $\Delta\Gamma$, the number of microstate inside the macrostate. When the time is very long, the distribution of probability in phase space tend to be constant at every allowed point. This justifie the microcanonical ensemble.

If energy fluctuate, you need an extra asumption: the system need to be big and far from critical points. So, the fluctuation of an extensive quantity like the energy $E$ goes like $\sim \sqrt{N}$, and the expectation $\Delta E$ like $\sim N$, when the number of subsystems $N$ is big. This means that the distribution of probability in energy is very narrow ($\Delta E\ll \bar{E}$).

The last assumption means that the distribution of probability in energy is approximately constant at $\bar{E}$ with width $\Delta E$ and then zero every were. Then, we can define $\Delta \Gamma$ as:

$$ \Delta \Gamma = \frac{d \Gamma (\bar{E})}{dE}\Delta E $$

where $\Gamma (E)$ is the number of states bellow energy $E$, and \frac{d \Gamma (E)}{dE} the density of states.

Then you can define entropy as $k_B\ln (\Delta\Gamma)$. Note that the entropy is not dependent of the precisely shape of the distribution of probability. When you can't make this approximations, means that entropy is strongly dependent of the shape of the probability distribution.