# Derive Canonical Ensemble from Maximum Entropy Principle

Consider a quantum system with Hamiltonian $$H$$, described by a density operator $$\rho$$. It is known that the expectation value of this system is

$$\langle H \rangle \equiv tr(\rho H) = E$$

I want to determine the density operator $$\rho$$ by the principle of maximum entropy, without the need to introduce a large number imaginary copies (ensemble) of the system (this is the route followed by Pathria and Beale). Here the entropy should be the von Neumann entropy of the density operator

$$S(\rho) = tr(-\rho \ln \rho)$$

Then I need to minimize $$-S(\rho) = \rho \ln \rho$$ under the following constraints:

• $$E = tr(\rho H)$$
• $$\rho$$ is a density operator, i.e. it is positive definite and $$tr(\rho) = 1$$.

Is it possible to follow this logic and finally arrive at

$$\rho = \frac{1}{Z} e^{-\beta H} \quad ?$$

Here $$\beta = 1/T$$ is the temperature and $$Z = tr(e^{-\beta H})$$ is the partition function.

Up to now I tried the following: construct the Lagrange function

$$\Lambda(\rho, \lambda) \equiv -S(\rho) + \lambda_1 (tr \rho - 1) + \lambda_2 (tr \rho H - E)$$

where $$\lambda_i$$'s are Lagrange multipliers. The minimization condition is

\begin{align*} \frac{\delta \Lambda}{\delta \rho} &= tr \frac{\delta}{\delta \rho} ( \rho \ln \rho + \lambda_1 \rho + \lambda_2 \rho H ) \\ &= tr(1 + \ln \rho + \lambda_1 + \lambda_2 H) \overset{!}{=} 0 \end{align*}

Thus

$$\rho = \exp(-1 - \lambda_1 - \lambda_2 H)$$

But then

• The variation $$\delta(\rho \ln \rho)/\delta \rho$$ may not be a simple generalization of $$d(x \ln x)/dx = 1 + \ln x$$, since $$\rho$$ is now a linear operator (can be imagined as a matrix, and I am now varying the matrix elements).
• I was sloppy to say that $$tr(1 + \ln \rho + \lambda_1 + \lambda_2 H) = 0$$ implies $$1 + \ln \rho + \lambda_1 + \lambda_2 H = 0$$.
• How to incorporate the constraint that $$\rho$$ is positive definite?
• How to related the Lagrange multipliers $$\lambda_1, \lambda_2$$ to the temperature $$T = 1/\beta$$? (I hope that this can be done without too much classical thermodynamics.)

A good way to start the proof is to first select some basis such that the density matrix $$\rho=\sum_{j}{p_j|\psi_j\rangle \langle\psi_j|}$$ where $$p_j>0$$ and $$\{|\psi_j\rangle\}$$ is orthonormal. This form can always be derived for any density matrix $$\rho$$ since we already assume $$\rho$$ to be positive-definite, which implies that it is Hermitian and diagonalizable. Also, by $$\text{tr}(\rho)=1$$, $$\sum_{j}{p_j}=1$$ as expected.

Now look back to the minimization condition you derive, $${\delta \Lambda \over \delta \rho}=\sum_{j}{\partial p_{j}(1+\log{p_j}+\lambda_1+\lambda_2 \langle\psi_j| H |\psi_j\rangle)}=0$$ Therefore, given $$Z=\sum_{j}{e^{-\lambda_2 \langle\psi_j| H |\psi_j\rangle}}$$, $$p_j={1 \over Z}{e^{-\lambda_2 \langle\psi_j| H |\psi_j\rangle}}$$ But let us inspect what our minimization condition means now: We set $$\{|\psi_j\rangle\}$$ to be some fixed basis, and by varying $$p_j$$ under constraints of $$\sum_{j}{p_j}=1$$ and energy conservation, we minimize the negative value of entropy $$-S$$. Therefore, there are choices of basis which can further vary $$S$$, so which basis can maximize $$S$$?

To find the answer, we recompute $$S$$ \begin{align} S & = -\sum_{j}{p_j\log{p_j}} \\ & = -\sum_{j}{\big(p_j(-\lambda_2 \langle\psi_j| H |\psi_j\rangle)-p_j\log{Z}\big)} \\ & = \lambda_2 E + \log{Z} \end{align} If we take a differential on both sides \begin{align} \delta S & = \delta\lambda_2 E + \delta(\log{Z}) \\ & = \delta\lambda_2 E - \delta\lambda_2\sum_{j}{\langle\psi_j| H |\psi_j\rangle e^{-\lambda_2 \langle\psi_j| H |\psi_j\rangle} \over Z} + \delta_{\{|\psi_j\rangle\}}(\log{Z}) \\ & = \delta_{\{|\psi_j\rangle\}}(\log{Z}) \end{align} where $$\delta_{\{|\psi_j\rangle\}}(\log{Z})$$ represents the variation of $$\log{Z}$$ when we vary only the basis $$\{|\psi_j\rangle\}$$ and keep $$\lambda_2$$ the same.

So we are left to maximize $$Z=\sum_{j}{e^{-\lambda_2 \langle\psi_j| H |\psi_j\rangle}}$$ with respect to the basis $$\{|\psi_j\rangle\}$$ while considering $$\lambda_2$$ to be a constant. Since exponential function is convex, we have the inequality (you can think it as a generalization for operator version of Jensen's inequality) $$\sum_{j}{e^{-\lambda_2 \langle\psi_j| H |\psi_j\rangle}} \leq \sum_{j}{\langle\psi_j|e^{-\lambda_2 H} |\psi_j\rangle}$$ where the equality holds when $$|\psi_j\rangle$$ is an eigenvector of $$H$$ for any $$j$$. Therefore, $$Z$$ is maximized at $$Z=\sum_{j}{e^{-\lambda_2 E_j}}$$ where $$E_j$$'s make the energy spectrum of $$H$$, and the density matrix at thermal equilibrium is $$\rho={1 \over Z}{e^{-\lambda_2 H}}$$ as you expect. Now let me answer the questions I have not replied:

1. How to incorporate the constraint that $$\rho$$ is positive definite?

The property that a matrix is positive-definite is hard to incorporate directly in the Lagrange's method. The Lagrange's method takes care of constraints which are equalities while the condition that $$p_j>0$$ gives inequalities. However, the density matrix at thermal equilibrium is naturally positive-definite since we are not varying $$p_j$$'s with respect to some random function, we are maximizing some carefully crafted function, entropy, $$S$$. It turns out $$p_j$$'s are always bigger than $$0$$ no matter what $$H$$ and $$\lambda_2$$ are.

1. How to relate the Lagrange multipliers $$\lambda_1$$, $$\lambda_2$$ to the temperature $$T={1 \over \beta}$$? (I hope that this can be done without too much classical thermodynamics.)

As you can see in our results, $$\lambda_1$$ disappears in $$p_j$$ and $$S$$ since $$\lambda_1$$ serves as a bias (the background energy that already exists and does not influence the statistics of our system). The more interesting part is $$\lambda_2$$, and let us denote $$\lambda_2 = \beta$$.

Historically there are two versions of entropy: The first one is the Clausius' version $$\delta S_C=T \delta Q$$, and the second is the Gibb's version $$S_G=-\sum_{j}{p_j\log{p_j}}$$ (Boltzmann's entropy about microstates is a special case of Gibb's). The first version of entropy comes with the usual temperature $$T$$, and the second version, as we can see previously, yields $$\beta$$ naturally. Here comes the crucial point: Starting with $$S_C$$, we can construct the first version equations for laws of thermodynamics in which $$T$$ is used, and similarly, a second version of equations where $$\beta$$ is used can be formulated. These two approaches to thermodynamics are both valid but just respectively from the macroscopic viewpoint and microscopic one. Therefore, the two versions of equations should be equivalent, and by comparison, we will have $$\beta={1 \over T}$$ and $$\delta S_C = \delta S_G$$. There is a quite explicit post about how this is done.

• Thanks for the detailed answer! One thing that may be improved: you separated the variation of $\rho$ to two steps: first vary the eigenvalues without affecting the eigenvectors of $\rho$, then show that the maximum entropy is achieved when the eigenvectors coincides with those of $H$. I think a more careful calculation of the matrix derivative $\delta \Lambda/\delta \rho$ may lead to $\rho \propto e^{-\lambda_2 H}$ in one step. Jun 26, 2022 at 7:28
• In addition, after obtaining $S = \lambda_2 E + \ln Z$, you said "so we are left to maximize $\ln Z$". What about $\lambda_2$? Why can it be treated as a constant here? Jun 26, 2022 at 9:04
• It is a great question! The reason why we can treat $\lambda_2$ as a constant here since we have the following argument. Consider some basis $\{|\psi_j \rangle\}$ which does not maximize $Z$. There is some small rotation $I+\delta U$ acting on the basis $\{|\psi_j \rangle\}$ such that the new basis $\{|\psi'_j \rangle\}$ will result in same $\lambda_2$ while having slightly larger $Z$. This is intuitively true, and notice the argument breaks down when we already have some basis maximizing $Z$. Jun 26, 2022 at 14:06
• @ZhengyuanYue I have found a much more mathematically rigorous way to prove that we can treat $\lambda_2$ as a constant, and the content has been updated. Thank you for your question since I have not thought about this aspect in such a deep way before. Jun 26, 2022 at 18:06
• I now have a quick way to show $\lambda_2 = \beta$: recall that in thermodynamics $dE = TdS$, or $dS = \beta dE$. You derived $S = \lambda_2 E + \ln Z$; meanwhile, $\ln Z$ is independent of $E$; therefore $dS = \lambda_2 dE$. By comparison, we get $\lambda_2 = \beta$. But then I wonder if I can directly motivate the definition of $T$ without resort to thermodynamics; for example, by separating the system into two parts that can exchange energy freely, and derive the density operator for the two parts; then argue that there is something equal in the two $\rho$'s. Jun 27, 2022 at 2:36