Entropy as a function of internal energy in an arbitrary ensemble

Question

The Boltzmann formula for entropy, $S_B=k\ln \Omega(E)$ holds in the microcanonical ensemble, where $E$ is fixed. In other ensembles, entropy is given by the Gibbs/Shannon formula: $S_G=-k\sum_i P_i \ln P_i$, where $i$ labels the microstates and $P_i$ is the probability of state $i$. Here energy can fluctuate and the internal energy is given by $U = \sum_i P_i E_i$. In the thermodynamic limit, where the system is infinitely large, the density of microstates around the most probable energy $E^*$ is narrow so $U \approx E^*$ and it doesn't matter whether one uses $S_B(E^*)$ or $S_G$ for the entropy.

From thermodynamics we know that the entropy is a function of the internal energy, $S=S(U)$ which can be seen approximately from the Boltzmann entropy, since $S_B(E^*) \approx S_B(U)$ in the thermodynamic limit.

I believe, however, that $S=S(U)$ should hold even in the case of small systems, well below the thermodynamic limit. One can imagine $n$ identical systems (replicas), isolated from each other but brought into contact with a reservoir (heat/particle bath, for example). In the case of large $n$, the system composed of the $n$ replicas is in the thermodynamic limit. Since the replicas don't interact with each other, their microstates are independent of each other and the entropy of the $n$-replica system is $S_n=n S_0$. The internal energy is $U_n = n U_0$.

Since the $n$-replica system is in the thermodynamic limit, it is true that $S_n=S_n(U_n)$. The $n$ replicas are independent of each other, therefore $S_0=S_0(U_0)$, thus the entropy should be a function of the internal energy even for small systems, regardless of the type of the reservoir (i.e. type of the ensemble). What I fail to see is, how to prove this in an exact manner. Starting from the expression of entropy $S_G=-k\sum_i P_i \ln P_i$ and internal energy $U = \sum_i P_i E_i$ one should be able to reach to an expression $S_G=S_G(U)$, without assuming anything about the narrowness of the density of states.

Edit: I think I was wrong about the fact that any small system can be thought of as being in the thermodynamic limit considering many replicas of the system. This seems to be true for systems composed of non-interacting particles, however, for systems of interacting particles boundary effects are important. At infinitely large systems boundary effects are negligible, but with an infinite number of small systems, they are generally not. Then the energy is not simply the system energy + reservoir energy because interactions with the reservoir are not negligible.

Answer 1

What you seem to want to do is to get $S_G(U)$ from canonical ensemble framework. This is possible, but tedious in practice.

Thermodynamic energy $U$ enters the canonical framework as constraint on the probabilities:

$$ U = \sum_i p_i^* E_i. $$

Since the canonical probabilities $p_i^* = \frac{e^{-\beta E_i}}{Z}$ depend on temperature, so does $U$.

In canonical ensemble, everything depends on temperature. So the customary first calculation is that of free energy $F$. When $F(T)$ is defined as $U(T)-TS_G(T)$, it can be shown from the Boltzmann probabilities that

$$ F(T) = -k_B T \ln Z, $$ where $$ Z(T) = \sum_ie^{-\frac{E_i}{k_BT}}. $$ So one usually tries to express this $Z$ in some convenient form. In simple cases, this function can be found as simple expression involving $T$, but in general not. Then we can find it numerically.

If we start with definition of entropy

$$ S_G(T) = -k_B \sum_i p_i^*\ln p_i^* $$ we can express this also by differentiating $Z$:

$$ S_G(T) = -\frac{\partial (-k_B T \ln Z)}{\partial T};~~~(*) $$ for a simple system with single phase, this should always work.

This way, we obtain $S_G$ as a function of $T$. So we need to eliminate $T$ in favor of $U$.

The function $U(T)$ can be found similarly to $S(T)$. If we start with

$$ U = \sum_i E_i \frac{e^{-\beta E_i}}{Z} $$ it is possible to express this as $$ U(T) = -\frac{\partial \ln Z}{\partial \beta} $$ so we can reuse the first calculation of $Z(T)$ and just differentiate.

For all systems that thermodynamics applies to, function $U(T)$ (everything but $T$ is assumed to be fixed) is strictly increasing, so in principle one can always invert it to get $T(U)$ and then use this in (*) to obtain $S_G(U)$. I think I saw an explicit calculation along these lines for ideal gas once - it worked, the same as Boltzmann entropy function (up to some negligible terms) was obtained, but it was quite demanding.