Maximizing entropy with Lagrange multipliers

Question

This is a problem I saw in a stat mech textbook, and I think it is a fairly common problem.

Given the entropy function: $$S = - \sum_{i=1}^N p_i \log p_i$$ Maximize $S$ subject to constraints:

$$ \sum_{i=1}^N p_i = 1 \\ \sum_{i=1}^N p_i e_i = c$$

It was suggested to solve this problem using Lagrange multipliers. So this is how I went about it: $$L(p,\lambda, \mu) = - \sum_{i=1}^N p_i \log p_i - (\lambda \sum_{i=1}^N p_i -1)- (\mu \sum_{i=1}^N p_i e_i - c) $$

$$\frac{\partial L}{\partial p_k} = -(\log p_i +1) - \lambda - \mu e_i = 0$$ A little arithmetic gives:

$$p_i = e^{-\lambda - \mu e_i -1}$$

Then I used the above constraints to solve for $p_i$.

$$\sum_{i=1}^N p_i = \frac{\sum e^{-\mu e_i}}{e^{\lambda+1}} = 1 \implies e^{\lambda +1} = \sum e^{-\mu e_i} $$

And

$$\sum_{i=1}^N e_i p_i = \frac{\sum_{i=1}^N e_i e^{-\mu e_i}}{e^{\lambda +1}} = c$$

Since I am not sure how to solve this final constraint and get a value for $\mu$, I said

$$p_i = \frac{e^{-\mu e_i}}{\sum_i e^{-\mu e_i}}$$

My question is, how do I solve for $\mu$?

Answer 1

You cannot solve for $\mu$ unless you know the $e_i$'s. But that shouldn't bother you, because in the context of the canonical ensemble, $\mu$ is defined to be the (inverse) temperature, and all quantities are written in terms of it. $c$ is the expected energy for a given $\mu$.

Answer 2

One may apply a following trick, let $f(\mu) = \sum e^{-\mu e_i}$, then: $$ \frac{\partial f}{ \partial\mu} = -\sum e_i e^{-\mu e_i} $$ the constraints lead to following differential equation: $$ \frac{\partial f}{ \partial\mu} = -cf \qquad f(0) = N $$ Where $N$ is number of operands in $e_i$. Which has a solution: $$ f(\mu) = N e^{-c \mu} $$ However, in general, there is no way to resolve the equation for $\mu$ in a closed form: $$ \sum e^{-\mu e_i} = N e^{-c \mu} $$ One may get a solution by some numerical technique.