Boltzmann distribution derivation from Lagrange Multipliers

Context

In the derivation of the Boltzmann factor and the canonical partition function based essentially on Lagrange multipliers presented here, the equalities, \begin{align*} p_j &= \frac{1}{Z} e^{\frac{\lambda_2 E_j}{k_B}} \\ Z &= \sum_j e^{\frac{\lambda_2 E_j}{k_B}} \end{align*} are established, where $$E_j$$ refers to the energy of a microstate, $$k_B$$ is the Boltzmann constant, and $$\lambda_2$$ is the remaining undetermined multiplier. Then, using the Gibbs entropy $$S = - k_B \sum_j p_j \ln(p_j)$$ along with some algebra, $$\begin{gather*} S = -\lambda_2 U + k_B \ln(Z) \end{gather*}$$ Finally, the article suggests the derivative, $$\begin{gather*} \frac{\partial S}{\partial U} = - \lambda_2 \end{gather*}$$ without further work. Note that the article also improperly uses a total derivative instead of a partial here as the whole point is to make the identification $$\frac{\partial S}{\partial U} = - \lambda_2 = \frac{1}{T}$$.

My Question

My question comes from the details of working out the partial derivative. I obtained, \begin{align*} \frac{\partial S}{\partial U} &= - \lambda_2 - U \frac{\partial \lambda_2}{\partial U} + \frac{k_B}{Z} \frac{\partial Z}{\partial U} \\ &= - \lambda_2 - U \frac{\partial \lambda_2}{\partial U} + \frac{1}{Z} \sum_j e^{\frac{\lambda_2 E_j}{k_B}} \left[ E_j \frac{\partial \lambda_2}{\partial U} + \lambda_2 \frac{\partial E_j}{\partial U} \right] \\ &= - \lambda_2 + \frac{1}{Z} \sum_j e^{\frac{\lambda_2 E_j}{k_B}} \lambda_2 \frac{\partial E_j}{\partial U} \end{align*} where we have used the statistical definition of $$U$$ and the equalities above to cancel one of the derivative terms involving $$\frac{\partial \lambda_2}{\partial U}$$. It is at this point that I have a question. Clearly, for this to work, we must have $$\frac{\partial E_j}{\partial U} = 0$$. But I cannot seem to really justify this step to myself. Is it simply that the energies of the particular microstates do not explicitly depend on the average energy? How would one justify this?

• First off, a similar version also holds for non-physical applications of the Max-Ent principle. Second, yes, why do you think the micro states depend on $U$? I mean they are fixed by the kind of system you consider and "external" parameters like $N,V,\ldots$. Once these things are fixed, you obtain $U$ as the average of the energies with respect to some probability distribution. Commented Sep 15, 2023 at 6:27
• I don’t think they depend on U, but I’m not convinced I can just assert that derivative vanishes in general. Commented Sep 15, 2023 at 11:01
• This statement is contradictory. Commented Sep 15, 2023 at 11:01
• BTW: It is a basic fact in the theory of Lagrange multipliers that the derivative of the constrained (say) maximum function with respect to a constraint is the corresponding multiplier... See e.g. here. Commented Sep 15, 2023 at 11:03
• Could you clarify your remark on the cost function? How does that apply to this context? Commented Sep 15, 2023 at 11:05

Clearly, for this to work, we must have $$\frac{\partial E_j}{\partial U} = 0$$. But I cannot seem to really justify this step to myself. Is it simply that the energies of the particular microstates do not explicitly depend on the average energy? How would one justify this?

This is because when we write $$\frac{\partial S}{\partial U}$$ with the intention of getting inverse of thermodynamic temperature, we have to assume that all the other thermodynamic state variables that $$S$$ depends on are constant; otherwise the derivative need not be function of temperature. In thermodynamics, $$S$$ is function of at least $$U,V,N$$, so the derivative is actually

$$\frac{\partial S}{\partial U}\bigg{|}_{V,N} = \frac{1}{T}.$$

Assuming constant $$V,N$$ in quantum theory implies constant Hamiltonian, and thus constant energy levels $$E_j$$. Changing $$U$$ in this partial derivative is thus allowed to happen only via changing the probabilities $$\rho_j$$'s, not via changing the energy levels $$E_j$$'s.

• That connection back to quantum stat mech is a really good one. Thanks for your answer! Commented Sep 16, 2023 at 17:23

Imo, your doubt is related to the assumption that the partial derivative of the energy of a microstate, $$(E_j$$), with respect to the total energy, $$(U$$), is zero, i.e., $$(\frac{\partial E_j}{\partial U} = 0$$).

In the context of statistical mechanics, the assumption $$(\frac{\partial E_j}{\partial U} = 0$$) is essentially an assumption about the statistical ensemble you are working with. Specifically, it assumes that the energy levels of the microstates are not influenced by the total energy of the system, (U). This assumption is valid for certain types of ensembles, such as the microcanonical ensemble or the canonical ensemble.

The microcanonical ensemble describes an isolated system with a fixed total energy, (U). In this ensemble, the energy of each microstate $$(E_j$$) is a constant because the total energy is fixed, and the system cannot exchange energy with its surroundings. Therefore, $$(\frac{\partial E_j}{\partial U} = 0$$) in this ensemble.

The canonical ensemble describes a system in contact with a heat bath at a constant temperature, $$(T$$). In this ensemble, the energy levels $$(E_j$$) of the microstates are typically assumed to be fixed properties of the system and not influenced by the total energy $$(U$$). This assumption is reasonable when dealing with macroscopic systems where the interactions between particles are weak, and the energy levels do not significantly change with variations in the total energy.

So, the assumption $$(\frac{\partial E_j}{\partial U} = 0$$) is justified in the context of these ensembles. Buuuut, it's important to note that this assumption might not hold in all situations. For systems with strong interactions or where the energy levels are explicitly dependent on the total energy, this assumption may not be valid, and you would need to consider more complex models.

• Why should the energies be (functionally) dependent on the average energy? Commented Sep 15, 2023 at 9:01
• I like this answer, especially since the idea that the microstates are “levels” really emphasizes why averages over them should not cause them to change. I think this statement is pretty much given in the microcanonical ensemble, but it’s a little trickier in the canonical. Does this effectively just say that the position of the level for a given micro state is invariant to how much energy your system happens to have? If so, I feel slightly foolish for my question. Commented Sep 15, 2023 at 11:08
• "For systems with strong interactions or where the energy levels are explicitly dependent on the total energy, this assumption may not be valid, and you would need to consider more complex models." Can you give an example for a system where this assumption is not valid? Commented Sep 15, 2023 at 16:04
• @ACuriousMind The assumption may not be valid is in strongly interacting quantum systems, particularly in strongly correlated electron systems in condensed matter physics. In such systems, the energy levels of individual particles (e.g., electrons) can depend on the total energy of the system, leading to nontrivial and energy-dependent interactions between particles. I'll ellaborate on it. Commented Sep 15, 2023 at 16:28
• In some materials, particularly in certain types of high-temperature superconductors and strongly correlated electron systems, the interactions between electrons are so strong that they significantly affect the energy levels and behaviour of individual electrons. In these materials, the energy levels of electrons may depend on the total energy of the system due to the strong electron-electron interactions. This energy dependence can lead to the formation of exotic phases of matter, such as Mott insulators, where the system behaves as an insulator even when it should nominally be conducting. Commented Sep 15, 2023 at 16:29