# Prove that entropy increases with the number of states

According to the formulation of entropy as $$S = -\sum_i P(i)\log(P(i))\quad,$$ how do we know that entropy increases with the number of states of a system regardless of their probability distribution? We can show this pretty easily for a system with a uniform distribution of states: $$S = -N(\frac{1}{N})\log(\frac{1}{N}) = -\log(\frac{1}{N})$$ Is there a way to show that this is the case for any arbitrary probability distribution?

• I don't know if I understand: If $p_0 =1$ and $p_i=0$ for $i \neq 0$, then $S=0$, irrespective of $N$. So take first $q_i=1/N$ for all $i$ and a fixed $N_q=N$, then pick $p_i$ as described above with $N_p=N+1$. The entropy associated to the $q_i$, $S_q=\log N$, is larger than the entropy associated to the $p_i$, $S_p=0$, although $N_q < N_p$. Do I miss something here? Could you give a reference of this statement you're referring to? Feb 4 at 20:58

If you have an isolated system, you can use Boltzmann's entropy formula $$S = k_{b}log(W)$$ where W is the number of accessible microstates. You can see that here, as the number of accessible states increases, the entropy increases as well.

Now for any given system (isolated or not), you can find an isolated system that encapsulates it. For that "bigger" system you can use the above logic to show that entropy increases as number of accessible microstates increase.

As stated the question does not work out. When doing statistical physics a system does not have an entropy, an ensemble has! (We get around this in macroscopic thermodynamics by assuming that the average over time for a system in equilibrium is the same as the ensemble average, or by thinking of the different parts of the system as the ensemble).

So you can't ask the question whether the system's entropy increases as you increase the number of states, it is just an invalid question.

You can ask how the entropy in a given ensemble of states changes as you increase the system size, but you have to define the ensemble (that is how the systems are prepared) as well.

For example, consider an ensemble when you assume uniform distribution over all accessible states. Then you are in the microcanonical ensemble. (And usually let your prepared energy $$E$$ increase proportionally to system size so that the limit makes sense).

Another possibility is to consider the canonical ensemble, where your system is couple to a thermal bath of a given temperature $$T$$. In this case the statement is also true in the thermodynamic limit.

It is not in general true for arbitrary non-equilibrium ensembles. E.g. you can keep the entropy constant by preparing the system in the state $$\rho = \lvert 0 \rangle \langle 0 \rvert$$ as your number of states increases.

• Yes, for macroscopic systems (and under the assumption that the time average of the fluctuations is the ensemble average). You can't measure the entropy of a single atom coupled to a thermal bath (without measuring its state again and again – which is equivalent to considering an ensemble). Feb 4 at 21:11