# Verifying a formula for the entropy of a system

I would like to show that the equation

$$S = -k\sum_{i = 1}^{N} p(n_{i}) \ln p(n_{i}),$$

holds for a binary system with a given definitinon of entropy.

To avoid confusion with $$N$$, let $$M$$ be the number of particles. Suppose that each particle can be in one of 2 energy states, $$\epsilon_1$$, and $$\epsilon_2$$. Suppose that we know the total energy of the system. Then we know exactly how many particles must be in each energy state. In that case, we will have $$\Omega = \frac{M!}{M_1!M_2!}$$, where $$M_1$$ is the number of particles in energy state $$\epsilon_1$$, and $$M_2$$ is the number of particles in energy state $$\epsilon_2$$.

This is the setup of our problem. We would now like to calculate the probability that a given particle is in state $$\epsilon_1$$. This is just $$M_1/M$$. Similarly, the probability that the particle is in state $$\epsilon_2$$ is $$M_2/M$$.

Therefore, for this particle,

$$S=-k\sum_{i=1}^{2} p_i\ln p_i = -k \left ( \frac{M_1}{M}\ln(M_1/M) + \frac{M_2}{M}\ln(M_2/M) \right )$$

$$= \frac{k}{M} \left ( M\ln M - M_1\ln M_1 - M_2\ln M_2 \right )$$

If we take the Stirling approximation for the entropy of the overall system, then we get:

$$S = k \ln \Omega = k \ln\frac{M!}{M_1!M_2!} \approx k \left[ M\ln{M} - M -(M_1\ln M_1 - M_1) -(M_2\ln M_2 - M_2) \right]$$

$$= k \left ( M\ln M - M_1\ln M_1 - M_2\ln M_2 \right )$$

But this is just $$M$$ times the expression for the entropy of a single particle! So we are done. The two expressions for entropy are almost exactly identical.

Note: I say almost exactly, and this is important. We took a Stirling approximation to get this equality, and so it doesn't hold exactly. The error came in when we assumed that all the particles were independent from each other, so we could just add their entropies together to get the total. However, the particles weren't exactly independent of each other, and so this step wasn't really formally justified. You'd have to argue that the particles were independent enough from each other that the approximation could be made in order for this proof to work formally.

• I'm afraid that I may have misunderstood your question completely. It seems like you are wanting to derive the formula that I started with from some other assumptions. If so, could edit your question a bit to specify the physical system you are thinking of in more detail? For example, are you thinking of a bunch of particles that can each be in one of two states? Also, what expression for entropy do you get to assume? S = k ln Omega? Commented Sep 23, 2018 at 22:07
• One of the expressions is the entropy for all the particles, one of them is for a single particle. Commented Sep 24, 2018 at 2:19