Maximizing entropy in QM (Sakurai's Modern Quantum Mechanics)

Question

There is section in Sakurai's "Modern Quantum Mechanics 2nd edition" page 188 that is quite confusing as to what he is doing. In the section on "Quantum Statistical Mechanics" he defines a quantity $\sigma = -\text{tr}(\rho \text{ln}\rho)$, where the entropy is given as $S = k \sigma$. Then in order to maximize $\sigma$ he states the following "Let us maximize $\sigma$ by requiring that $$\delta \sigma = 0.~~~~~~~~~~~~~(1)$$

However, we must take into account the constraint that the ensemble average of $H$ has a certain prescribed value. In the language of statistical mechanics, $[H]$ is identified with the internal energy per constituent, denoted by $U$: $$[H] = \text{tr}(\rho H) = U.$$ In addition, we should not forget the normalization constraint $\sum_{k} \rho_{kk} = 1$ (where $\rho$ is the density operator). So our basic task is to require (1) subject to the constraints $$\delta(H) = \sum_{k} \delta \rho_{kk} E_k = 0$$ and $$\delta(\text{tr} \rho) = \sum_{k} \delta \rho_{kk} = 0.$$

We can most readily accomplish this by using Lagrange multipliers. We obtain $$\sum_{k} \delta \rho_{kk}[(\text{ln}\rho_{kk} +1) + \beta E_{k} + \gamma] = 0$$ which for an arbitrary variation is possible only if $$\rho_{kk} = \text{exp}(- \beta E_{k} - \gamma - 1).$$"

Question: What is the reasoning for the requirement (1) in order to maximize $\sigma$, and how are the constraints obtained?

Answer 1

This answer has a bit more information theoretic flavor than it does QM, so I apologize in advance.

The goal here is to figure out the distribution $\rho$, knowing the average energy of the system, $\text{tr}(\rho H)=\langle H\rangle$, and that the distribution should be normalized, $\text{tr}(\rho)=1$.

Think of the entropy as being the amount of hidden information in the system. It quantifies how ignorant we are concerning the system in question. The distribution we want is the one that uses the known information, and is unbiased with regard to any other information. This is just another way of saying that we want to be as ignorant as possible while satisfying the constraints. We quantify this statement by maximizing our ignorance subject to constraints.

To get ahead we define the functional: $$ S[\rho,\alpha,\beta] = -k ~\text{tr}(\rho\ln\rho)+\alpha(1-\text{tr}(\rho))+\beta(\langle H\rangle-\text{tr}(\rho H)) $$ What does this functional mean? The first term is just the entropy (hidden information), while the next two terms are constraints. $\alpha$ and $\beta$ are Lagrange multipliers which enforce the constraints. Maximizing this functional will ensure that our distribution has the least bias possible.

First off, note that if the energy constraint was not there, then the maximizing distribution would be constant. Every state would be equiprobable. In essence, we would know nothing about the system because we would be completely ignorant of which states are more or less occupied.

Note that maximizing $S$ with respect to the LMs just gives the constraints back: $$ \begin{align} \frac{\delta S}{\delta\alpha}=0&\Rightarrow \text{tr}(\rho)=1\\ \frac{\delta S}{\delta\beta}=0&\Rightarrow \text{tr}(\rho H)=\langle H\rangle \end{align} $$ Maximizing with respect to the distribution gives us: $$ \frac{\delta S}{\delta\rho}=0\Rightarrow \rho = \text{diag}(e^{-\beta E_n-\alpha-1}) $$ We can now use the normalization constraint to define the partition function and get rid of the $\alpha$ LM. We also note that the LM coming from the energy constraint IS the inverse temperature. Standard relationships from statmech follow.

This procedure is very general. Let's say you have a bunch of data from commuting observables, so you can estimate a bunch of values, $\langle \mathcal O_i \rangle=\text{tr}(\rho\mathcal O_i)$. You define a general entropy functional as: $$ S[\rho,\vec\alpha]=-k ~\text{tr}(\rho\ln\rho)+\sum_i\alpha_i\left(\langle \mathcal O_i \rangle-\text{tr}(\rho\mathcal O_i)\right ) $$ You then maximize this to find the least biased distribution that satisfies the constraints coming from your data.