# Relation between entropy and number of particles

In all continuous classical systems, entropy is of the order N (Number of particles). In natural units:

$$\quad S \sim N \qquad \qquad$$

In such systems, number of microstates is proportional to phase space volume.

$$\quad \Omega_1 \sim V \cdot\text{(radius in momentum space)}^{\text{Degree of freedom}}$$

For no external potential, this radius has a simple relation with Total thermal energy.

$$\quad r \sim U^{1/2}$$

So for a monoatomic gas with $$N$$ atoms:

$$\quad \Omega = \text{f}(N) \cdot V^N \cdot U^{Nf/2}$$

But f(N) can be any complicated function. How can we show: $$S \sim N$$?

• Search term: extensive versus intensive variables.
– rob
Commented Jun 23, 2023 at 15:44

But $$f(N)$$ can be any complicated function. How can we show: $$𝑆\sim𝑁$$?

The answer is: you can't, because it's not, in general. Even in your own expression, with \begin{align} \Omega = f(N) V^N U^{Nf/2} \end{align} you have to ask: how do $$V$$ and $$U$$ scale with $$N$$? That's what the people pointing to "intensive" versus "extensive" variables are talking about. For example, if I double $$N$$ by copying the system and setting the copy next to it, how does the variable change (likewise, I could cut it in half)?

Tl;dr: you have to assume that the particles that make up the system are weakly interacting/independent. If you can do that, then you can derive the constraints on the form of $$f(N)$$, and those constraints will lead to $$S\sim N$$.

Not a full derivation, but here are some of the specifics:

The definition of entropy was given by Gibbs: \begin{align} S = -k_B \sum_{i} P_i\ln P_i, \end{align} where $$P_i$$ is the probability that the system is in the state labeled by $$i$$. Let's split the system into two parts, and give the two parts the labels $$j$$ and $$l$$. The formula becomes: \begin{align} S = -k_B \sum_{jl} P_{jl}\ln P_{jl}, \end{align} where $$P_{jl}$$ is what's known as a joint probability: the probability that the first state label that we can call $$J$$ has the value $$j$$, and the second state label $$L$$ has the value $$l$$ at the same time. Note that $$\sum_{jl} P_{jl} = 1$$, because probabilities must sum to 1, by definition.

There's something called conditional probability $$P_{j|l}$$ where we read this as, "The probability that $$J=j$$ (randomly) given that $$L=l$$ (non-randomly)." In equations it is defined by the relation to the joint probability by \begin{align} P_{jl} = P_{j|l} P_l, \end{align} where $$P_l$$ is called the marginal probability and it is defined by $$P_l = \sum_j P_{jl}$$.

Last definition. If $$P_{j|l} = P_j$$ (i.e. the conditional probability for $$j$$ doesn't depend on $$l$$) then we say that the random variables $$J$$ and $$L$$ are independent.

Now, let's put our definitions to use. First, we insert the definition for conditional probability into the formula for entropy: \begin{align} S &= -k_b \sum_{jl} P_{j|l}P_l \ln\left(P_{j|l}P_l\right) \\ &= -k_b \sum_{l}P_l\left(\sum_j P_{j|l}\left[ \ln P_{j|l} + \ln P_l\right]\right) \\ &= -k_b \left[\sum_{l}\left(P_l\sum_j P_{j|l} \ln P_{j|l}\right) + \sum_{l}\left(P_l\ln P_l\sum_j P_{j|l}\right)\right] \\ &\qquad \text{because }\sum_{j} P_{j|l} = 1: \\ &= -k_b \left[\sum_{l}\left(P_l\sum_j P_{j|l} \ln P_{j|l}\right) + \sum_{l}P_l\ln P_l\right] \\ &= S(j|l) + S(l), \end{align} where the last line is the definition of the conditional entropy $$S(j|l)$$. Note that if $$P_{j|l} = P_j$$ then $$S(j|l) = S(j)$$.

The necessary and sufficient condition for $$S\sim N$$ is this: if we label the states of the $$N^{\mathrm{th}}$$ particle with $$l$$ and the states of every other particle with $$j$$, then $$P_{j|l} = P_j$$, and that this holds for every $$N$$ between $$1$$ and the $$N$$ of interest (proof by induction). In other words, $$S\sim N$$ (entropy is extensive in $$N$$) when you can assume that the interactions between the particles are weak enough to be negligible.

It's an assumption that works very well most of the time. It's what we assume for the ideal gas. For solids, where the molecules interact quite strongly, you can either talk about how the number of particle on the surface boundary between divisions of the system in half are much fewer than the number of molecules in the bulk, or you can say that the particles of interest are the phonons (sound equivalent to photons) that travel through the solid as non-interacting waves.

The assumption runs into trouble when you can't make these kind of approximations. For example, a Coulomb gas (i.e. a plasma) has long-range interactions that make it a non-ideal gas and its entropy may not be extensive. You can also add long-range interactions to the "simple" Ising model. The results of doing that are being actively studied and are known to produce "entanglement entropy," the quantum mechanical version of non-independent conditional probability.

Multiplying a large number with very large number is insignificant—for example, $$10^{23} \cdot 10^{10^{23}} \sim 10^{10^{23}}$$. Both $$V^N$$ and $$U^{Nf/2}$$ are exponentials of large numbers and hence are very large. This means $$f(N)$$ must be very large, which fits $$\ln{f(N)} \sim N \ln g(N)$$. This results in an overall exponential function of $$N$$; that is, $$\Omega \sim (...)^N$$ implies $$S \sim N$$.

This argues that the form of $$f(N)$$ has to be an exponential with power $$N$$—that it cannot be any other type of function.

The dominant contribution is proportional to the number of particles. This is a particular case of a more general property: entropy is an extensive quantity. That is, entropy is proportional to the size of the system, which is largely characterized by the number of particles in it.

I don't know of a general proof for any molecule and any interaction but every system for which we can write analytic models shows this behavior. Here are two elementary examples:

Example 1: Lattice model with no interactions

Say we have a lattice of size $$N$$ that contains $$N$$ particles, then the number of configurations is $$\Omega(L,N) = \frac{L!}{N! (L-N)!}$$ Using Sterling's formula $$x! = x\ln x - x + O(\ln x)$$ the entropy is $$S = \ln\Omega = N \left( \frac{L-N}{N}\ln\frac{L-N}{L} \right)$$ The quantity inside the parenthesis is intensive (depends only on the ratio $$L/N$$, which makes $$S$$ proportional to $$N$$.

Example 2: Ideal gas in the canonical ensemble

The canonical partition function is $$Q = \frac{V^N} {N! \Lambda^{3N}}$$ where $$\Lambda = h/\sqrt{2\pi m k T}$$. Its log is $$\ln Q = N\left( \ln\frac{V}{N} + 1 - \ln\Lambda^3 \right)$$ Here again the quantity in parenthesis is intensive (it depends on the intensive ratio $$V/N=v$$ so $$\ln Q$$ is extensive. And since $$\ln Q = -(U-TS)/kT$$ and $$U$$ is extensive, we can carry this result to $$S$$.