I'm trying to understand this paper on Maximum Entropy by Jaynes, and am stuck on something which should be rather simple.

We're attempting to maximize the entropy $-\sum_i p_i \ln(p_i)$ subject to the constraints $\langle f \rangle = \sum_i p_i f_i$ and $\sum_i p_i = 1$. Using Lagrange multipliers, we have:

$$L = -\sum_i p_i \ln(p_i) + \lambda \left( \sum_i p_i f_i - \langle f \rangle \right) + \mu \left( \sum_i p_i - 1 \right)$$

$$\frac{\delta L}{\delta p_i} = - \ln(p_i) - 1 + \lambda f_i + \mu = 0$$

So $p_i = e^{-1 + \mu + \lambda f_i}$. Plugging this back into the definition for $\langle f \rangle$:

$$\langle f \rangle = \sum_i p_i f_i = \sum_i f_i e^{-1 + \mu + \lambda f_i} = \sum_i f_i e^{-1 + \mu + \lambda f_i}$$

Equation (2-5) in Jaynes' paper states (ignoring sign conventions) that $\langle f \rangle = \frac{\partial}{\partial \lambda} \ln{Z}$, where $Z$ is the partition function, probably defined as $Z = \sum_i e^{-1 + \mu + \lambda f_i}$, or just simply $Z = \sum_i e^{\lambda f_i}$ if we ignore the normalisation. I tried to check this:

$$\langle f \rangle = \frac{1}{Z} \frac{\partial}{\partial \lambda} Z = \frac{1}{Z} \frac{\partial}{\partial \lambda} \sum_i e^{\lambda f_i} = \frac{1}{Z} \sum_i f_i e^{\lambda f_i} = \frac{1}{Z} \langle f \rangle$$

Which I doubt is correct. I'm probably making some obvious mistake, but for the life of me I can't find it.

  • $\begingroup$ Minor comment to the post (v1): Please consider to mention explicitly author, title, etc. of link, so it is possible to reconstruct link in case of link rot. $\endgroup$
    – Qmechanic
    Commented Jun 6, 2015 at 23:25

2 Answers 2


You're almost there.

The thing to realise about the partition function is that it is the normalisation. If statistical mechanics were being developed today I'm sure its name would be "normalisation factor", but "partition function" has stuck historically.

Now, the partition function is in fact defined as $Z = \sum_i e^{\lambda f_i}$, so that $$ p_i = \frac{1}{Z}e^{\lambda f_i}. $$ This is the formula you should have tattooed on the inside of your brain if you want to think much about statistical mechanics. You can note that this implies $Z = e^{1-\mu}$, and if you like you can think of this as its definition.

With all this in mind, it should now be clear that your derivation is correct up to the last equality, which should instead say $$ \dots = \frac{1}{Z} \sum_i f_i e^{\lambda f_i} = \langle f \rangle. $$


Your mistake is in the last line. You assert:

\begin{equation*} \langle f \rangle=\sum_i f_i e^{\lambda f_i} \end{equation*}

but that isn't true, rather:

\begin{equation*} \langle f \rangle=\sum_i f_i e^{\lambda f_i-1+\mu} \end{equation*}

To convert correctly between your two expressions for $\langle f \rangle$, you must figure out a way to express $\mu$ (which is not the chemical potential, to avoid any potential confusion) in terms of other variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.