This is a problem I saw in a stat mech textbook, and I think it is a fairly common problem.

Given the entropy function: $$S = - \sum_{i=1}^N p_i \log p_i$$ Maximize $S$ subject to constraints:

$$ \sum_{i=1}^N p_i = 1 \\ \sum_{i=1}^N p_i e_i = c$$

It was suggested to solve this problem using Lagrange multipliers. So this is how I went about it: $$L(p,\lambda, \mu) = - \sum_{i=1}^N p_i \log p_i - (\lambda \sum_{i=1}^N p_i -1)- (\mu \sum_{i=1}^N p_i e_i - c) $$

$$\frac{\partial L}{\partial p_k} = -(\log p_i +1) - \lambda - \mu e_i = 0$$ A little arithmetic gives:

$$p_i = e^{-\lambda - \mu e_i -1}$$

Then I used the above constraints to solve for $p_i$.

$$\sum_{i=1}^N p_i = \frac{\sum e^{-\mu e_i}}{e^{\lambda+1}} = 1 \implies e^{\lambda +1} = \sum e^{-\mu e_i} $$


$$\sum_{i=1}^N e_i p_i = \frac{\sum_{i=1}^N e_i e^{-\mu e_i}}{e^{\lambda +1}} = c$$

Since I am not sure how to solve this final constraint and get a value for $\mu$, I said

$$p_i = \frac{e^{-\mu e_i}}{\sum_i e^{-\mu e_i}}$$

My question is, how do I solve for $\mu$?


One may apply a following trick, let $f(\mu) = \sum e^{-\mu e_i}$, then: $$ \frac{\partial f}{ \partial\mu} = -\sum e_i e^{-\mu e_i} $$ the constraints lead to following differential equation: $$ \frac{\partial f}{ \partial\mu} = -cf \qquad f(0) = N $$ Where $N$ is number of operands in $e_i$. Which has a solution: $$ f(\mu) = N e^{-c \mu} $$ However, in general, there is no way to resolve the equation for $\mu$ in a closed form: $$ \sum e^{-\mu e_i} = N e^{-c \mu} $$ One may get a solution by some numerical technique.


You cannot solve for $\mu$ unless you know the $e_i$'s. But that shouldn't bother you, because in the context of the canonical ensemble, $\mu$ is defined to be the (inverse) temperature, and all quantities are written in terms of it. $c$ is the expected energy for a given $\mu$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.