# Entropy Maximization using undetermined multipliers

A system can be in any one of N states. Using the method of undetermined multipliers to show that for the maximum entropy, $S = -k \sum_i p_i \ln p_i$ where $p_i = 1/N$, $$S = k \ln N\,,$$
the solution is as follows:

Using an undetermined multiplier $\alpha$, write $$f = -k \sum_i(p_i \ln p_i - \alpha p_i)$$ $$\frac{\partial f}{\partial p_j} = -k (\ln p_i + 1 -\alpha ) = 0$$

$\ln p_j = \alpha -1$for all $j$, therefore all $p_i$ are equal. The maximum entropy is $$S_{max} = -k \sum_i(\frac{1}{N}\ln \frac{1}{N}) = k \ln N$$

I don't understand the last part on how to go from the sum to the final expression, given $\sum_i p_i = 1\;.$

• This is not a homework problem. Who ever flagged it as such obviously did not read through the entire post and is quite possibly an idoit. This is from a problems and solution book! – mcodesmart Oct 12 '13 at 7:32
• Please see meta.physics.stackexchange.com/questions/714/… for details on when the homework tag applies. Although this isn't a homework question, the tag is correct in this case - but it doesn't mean it's a bad question. – Nathaniel Oct 12 '13 at 7:38

## 2 Answers

It's a very simple mathematical identity, which arises because the sum is over $N$ things that are all the same. So $$-k \sum_{i=1}^N \frac{1}{N}\ln \frac{1}{N}$$ is the same as $$-k N \left( \frac{1}{N}\ln \frac{1}{N} \right)$$ and I think you should be able to do the rest from there.

Since $p_j$ is same for all $j$, we have for the normalisation $\sum_j p_j = N*p_j = 1$. This implies $p_j = 1/N$.

Now, $S_{max} = -k \sum_j p_j \ln p_j = -k N * (p_j \ln p_j)$. Now substituting $p_j = 1/N$ should give the expression you are looking for.