# Maximizing entropy in QM (Sakurai's Modern Quantum Mechanics)

There is section in Sakurai's "Modern Quantum Mechanics 2nd edition" page 188 that is quite confusing as to what he is doing. In the section on "Quantum Statistical Mechanics" he defines a quantity $\sigma = -\text{tr}(\rho \text{ln}\rho)$, where the entropy is given as $S = k \sigma$. Then in order to maximize $\sigma$ he states the following "Let us maximize $\sigma$ by requiring that $$\delta \sigma = 0.~~~~~~~~~~~~~(1)$$

However, we must take into account the constraint that the ensemble average of $H$ has a certain prescribed value. In the language of statistical mechanics, $[H]$ is identified with the internal energy per constituent, denoted by $U$: $$[H] = \text{tr}(\rho H) = U.$$ In addition, we should not forget the normalization constraint $\sum_{k} \rho_{kk} = 1$ (where $\rho$ is the density operator). So our basic task is to require (1) subject to the constraints $$\delta(H) = \sum_{k} \delta \rho_{kk} E_k = 0$$ and $$\delta(\text{tr} \rho) = \sum_{k} \delta \rho_{kk} = 0.$$

We can most readily accomplish this by using Lagrange multipliers. We obtain $$\sum_{k} \delta \rho_{kk}[(\text{ln}\rho_{kk} +1) + \beta E_{k} + \gamma] = 0$$ which for an arbitrary variation is possible only if $$\rho_{kk} = \text{exp}(- \beta E_{k} - \gamma - 1).$$"

Question: What is the reasoning for the requirement (1) in order to maximize $\sigma$, and how are the constraints obtained?

• What do you mean by "how are the constraints obtained"? It says in your quote that one is due to the ensemble average being a fixed value, and the other comes from the general normalization condition. – Michael Seifert May 2 '17 at 20:04
• Requirement (1) is just 'the derivative is zero at the maximum'. – knzhou May 2 '17 at 20:06
• @knzhou What derivative? What exactly is $\delta$? How is it defined mathematically? – user101311 May 3 '17 at 17:52
• @MichaelSeifert Do you know what $\delta \sigma = 0$ means exactly? – user101311 May 3 '17 at 17:58
• $\delta$ here stands for the differential of a function. The entropy $\sigma$ is a function of the density matrix elements $\rho_{kk}$. – Michael Seifert May 3 '17 at 18:22

The goal here is to figure out the distribution $\rho$, knowing the average energy of the system, $\text{tr}(\rho H)=\langle H\rangle$, and that the distribution should be normalized, $\text{tr}(\rho)=1$.
To get ahead we define the functional: $$S[\rho,\alpha,\beta] = -k ~\text{tr}(\rho\ln\rho)+\alpha(1-\text{tr}(\rho))+\beta(\langle H\rangle-\text{tr}(\rho H))$$ What does this functional mean? The first term is just the entropy (hidden information), while the next two terms are constraints. $\alpha$ and $\beta$ are Lagrange multipliers which enforce the constraints. Maximizing this functional will ensure that our distribution has the least bias possible.
Note that maximizing $S$ with respect to the LMs just gives the constraints back: \begin{align} \frac{\delta S}{\delta\alpha}=0&\Rightarrow \text{tr}(\rho)=1\\ \frac{\delta S}{\delta\beta}=0&\Rightarrow \text{tr}(\rho H)=\langle H\rangle \end{align} Maximizing with respect to the distribution gives us: $$\frac{\delta S}{\delta\rho}=0\Rightarrow \rho = \text{diag}(e^{-\beta E_n-\alpha-1})$$ We can now use the normalization constraint to define the partition function and get rid of the $\alpha$ LM. We also note that the LM coming from the energy constraint IS the inverse temperature. Standard relationships from statmech follow.
This procedure is very general. Let's say you have a bunch of data from commuting observables, so you can estimate a bunch of values, $\langle \mathcal O_i \rangle=\text{tr}(\rho\mathcal O_i)$. You define a general entropy functional as: $$S[\rho,\vec\alpha]=-k ~\text{tr}(\rho\ln\rho)+\sum_i\alpha_i\left(\langle \mathcal O_i \rangle-\text{tr}(\rho\mathcal O_i)\right )$$ You then maximize this to find the least biased distribution that satisfies the constraints coming from your data.