Why is entropy an extensive quantity?

Question

If we have two identical isolated macroscopic systems both with energy $E$. The number of accessible states of each of them is $\Omega(E)$ and hence the entropy is $\ln\Omega(E)$. Now if we put them in thermal contact to form a larger isolated system with energy 2$E$ (suppose there are weakly interacting). Then the number of accessible states of the whole system is

$$\sum_x\Omega(x)\Omega(2E-x)$$ but not just $$\Omega(E)\Omega(E)$$

So the total entropy is $$\ln\sum_x\Omega(x)\Omega(2E-x)$$

But not just $$\ln[\Omega(E)\Omega(E)]=\ln\Omega(E)+\ln\Omega(E)=S_1+S_2$$.

So why do we say entropy is extensive?

Answer 1

I think there is a misunderstanding. You are perfectly right when you write that the total micro canonical entropy of a combined system will be

\begin{equation} S_\textrm{combined}(2E) = k_B\ln \sum_x \Omega(x)\Omega(2E-x) \end{equation}

The micro canonical entropy ought to be a function of only the total energy, total amount of matter and total volume of a system and the formula you gave does satisfy these requirements.

It also has to be extensive in the thermodynamic limit which means that it has to approach a Eulerian function of degree 1 i.e. such that for large system sizes we have $S_\textrm{combined}(2E) \sim 2S(E)$.

This can be shown by noticing that $\Omega(x) = e^{S(x)/k_B}$. If we further assume that $S(x) \sim Ns(x)$ in the thermodynamic limit (where $s(x) \sim \mathcal{O}(1)$) then: \begin{equation} S_\textrm{combined}(2E) \sim k_B\ln \sum_x e^{Ns(x)/k_B} \cdot e^{Ns(2E-x)/k_B} \end{equation} Now, because $N$ virtually tends to infinity, the sum will be overwhelmingly dominated by its most probable value that we call $x^*$ (in virtue of the saddle point theorem). If the two sub-systems are the same, the only possibility is that $x^* = E$ and we get: \begin{equation} S_\textrm{combined}(2E) \sim k_B\ln e^{Ns(x^*)/k_B} \cdot e^{Ns(2E-x^*)/k_B} \sim 2S(E) \end{equation} This is not an exact result for any finite system size. You can see it as an exact result in the ideal thermodynamic limit or as an approximate result for finite system sizes.

People performing numerical simulations in statistical mechanics (with a finite number of simulated objects) have to worry about this aspect all the time.

Answer 2

Major edit: In @gatsu's answer, it is pointed out that only the amount of energy should matter, which is correct, as there's no such thing as distinguishable microstates with only rearranged energy (think stars-and-bars-type entropy calculations). So, I've edited out that part of the first paragraph and equations (in the first draft, I dropped that part of the equation midway through without realizing that doing so invalidated my answer).

The combined entropy of identical systems in thermal contact is given by $$\sum_x \Omega(x)\Omega(2E-x).$$ But, for large $E$, this is sharply peaked at $x=E$, so the other terms don't matter. $$ S_\textrm{total} = \log \sum_x \Omega(x)\Omega(2E-x) \\ \approx \log \Omega(E)\Omega(E) \\ = 2\log\Omega(E) \\ = 2S $$

---

Addendum:

The above is more a demonstration that entropy is an extensive property, not an additive property (as originally asked). In fact, if systems $A$ and $B$ are not identical, then the total entropy will be greater than $S_A + S_B$ in general due to the interaction. This is another statement of the Second Law of Thermodynamics.

---

Addendum 2:

I've been asked to stop being such a physicist and put a little rigor in my math. Specifically, is the peak value of the combined entropy given by $$\sum_x \Omega(x)\Omega(2E-x)$$ prominent enough to ignore all other terms? Since any non-trivial examples of $\Omega(x)$ would prevent an analytic result, I'll just consider the easier $$\sum_x \binom{2E}{x} = 2^{2E}$$ (yup, still a physicist).

Using Stirling's approximation, $$ \log\binom{2E}{E} = \log\frac{(2E)!}{(E!)^2} \\ \approx 2E\log(2E) - 2E - 2\left(E\log E - E\right) \\ = 2E\log2 + 2E\log E - 2E - 2E\log E + 2E \\ = 2E\log2 $$ And therefore, $$ \binom{2E}{E} \approx e^{2E\log2} \\ = 2^{2E} $$ Which is what we found at the beginning of this addendum. Since $\Omega(x)\Omega(2E-x)$ is similarly peaked at $x = E$, the result should still hold.

---

Addendum 3:

Hang on, let's try a somewhat real entropy example: each system is made of $n$ particles that share $x$ quanta of energy. The entropy, after standard stars-and-bars analysis, is given by $$S = \log\Omega(x) = \log\frac{(n + x - 1)!}{x!(n-1)!}$$ The combined entropy is given by $$ S_{total} = \log\sum_{x=0}^{2E}\Omega(x)\Omega(2E-x) \\ = \log\sum_{x=0}^{2E}\left[\frac{(n + x - 1)!}{x!(n-1)!}\frac{(n + 2E - x - 1)!}{(2E-x)!(n-1)!}\right] \\ = \log\frac{(2E + 2n - 1)!}{(2E)!(2n - 1)!} \qquad \textrm{(courtesy of Wolfram Alpha)}\\ \approx (2E + 2n - 1)\log(2E + 2n - 1) - (2E + 2n - 1) - 2E\log 2E + 2E - (2n - 1)\log(2n - 1) + (2n - 1) \\ \approx 2E\log\left(1 + \frac{n}{E}\right) + 2n\log\left(1 + \frac{E}{n}\right) $$

Now, the largest term in that sum is when $x = E$: $$ S_{total} \approx \log(\Omega(E)^2) \\ = 2\log\left(\frac{(n + E - 1)!}{E!(n-1)!}\right) \\ =2((n + E - 1)\log(n + E - 1) - (n + E - 1) - E\log E + E - (n-1)\log(n-1) + (n-1) \\ \approx 2E\log\left(1+\frac{n}{E}\right) + 2n\log\left(1+\frac{E}{n}\right) $$ which is the same as the full sum.

Answer 3

So why do we say entropy is extensive?

It is a convention that is possible and useful for weakly interacting systems. Multiplicity of state of the system made of two systems of the same kind and size and energy $E$ is

$$ \Omega'(2E) = \Omega(E)\Omega(E) $$

with very good accuracy, the other terms in the sum over $x$ you indicated can be neglected. Entropy can be defined as $\log \Omega$ and then it is extensive - the higher the greater the number of particles in the system.

For strongly interacting systems or systems with very low number of particles, the other terms in the sum for total multiplicity are not negligible and statistical physics is not applicable in this way.

Answer 4

Additional heuristic arguments.

In thermal equilibrium, each subsystem assume value that maximize the probability, say with energy $U$ (with fluctuation that vanishes in the thermodynamic limit)

$$S_1 = \ln \Omega_1(U),\quad S_2=\ln\Omega_2(E-U)$$

The composite system is a sum over energy values

$$S_C = \ln\left\{\sum_{x=0}^{E}\Omega_1(x)\Omega_2(E-x)\right\}$$

This sum is larger than its largest term, but smaller than the maximum term times the number of terms (say $E/\delta E$)

$$\Omega_1(U)\Omega_2(E-U)\le \sum_{x=0}^{E}\Omega_1(x)\Omega_2(E-x)\le \frac{E}{\delta E}\Omega_1(U)\Omega_2(E-U)$$

Taking the logarithms, we have

$$S_1+S_2\le S_C\le S_1+S_2 + \ln E - \ln\delta E$$

The last two terms are of order $\ln N$, where as others are of order $N$; therefore, in the thermodynamic limit, we have the strict additivity of entropy

$$S_C\rightarrow S_1+S_2$$

See Silvio Salinas (2001), Introduction to Statistical Physics.