Microcanonical ensemble through Maximum Entropy method

Question

The entropy functional over the phase-space is given by:

$$ S[\rho(\Gamma)] = - \int \rho(\Gamma) \ln (\rho(\Gamma)) d\Gamma $$ where $\rho(\Gamma)$ is the probability density of the system over phase-space $\Gamma$.

To impose the constraint that $\rho(\Gamma)$ is normalized, we introduce a Lagrange multiplier $\lambda$ and form the modified functional $ \tilde{S}[\rho(\Gamma)] $: $$ \tilde{S}[\rho(\Gamma)] = S[\rho(\Gamma)] + \lambda \left( \int \rho(\Gamma) d\Gamma - 1 \right) $$ Here, the term $\lambda \left( \int \rho(\Gamma) d\Gamma - 1 \right)$ enforces the normalization condition: $$ \int \rho(\Gamma) d\Gamma = 1 $$

The functional derivative is defined according to $$ \int d\Gamma \frac{\delta \tilde S\left[\rho(\Gamma)\right]}{\delta \rho(\Gamma)} \eta(\Gamma) d\Gamma := \frac{d \tilde S\left[\rho(\Gamma) + h \eta(\Gamma) \right]}{dh}\Bigg|_{h=0} $$ where $\eta(\Gamma)$ is any arbitrary test function.

The condition of Maximum Entropy requires finding the probability density $\rho_e(\Gamma)$ which extremises the entropy. Therefore, it can be mathematically represented as,

$$ \frac{\delta \tilde S\left[\rho(\Gamma) = \rho_e(\Gamma)\right]}{\delta \rho(\Gamma)} = 0 $$

From the above equation, we can calculate that $$ \frac{\delta \tilde S\left[\rho(\Gamma)\right]}{\delta \rho(\Gamma)} = \ln \{ \rho(\Gamma) \} + \lambda + 1 $$

Setting the above value to zero, at $\rho(\Gamma) = \rho_e(\Gamma)$, we get $\rho_e(\Gamma) = e^{-(\lambda + 1)}$. The main highlight is that the probability is independent of the phase-space point $\Gamma$ and therefore will be a constant uniform distribution for all the possible states. Further applying the normalization condition, we can show that

$$ \rho_e(\Gamma) = \frac{1}{\int d\Gamma} = \frac{1}{\Omega} $$

where $\Omega$ is the volume of the phase-space or the total number of microstates.

Question

In principle, the microcanonical ensemble is constrained to have an energy, E and therefore, the probability density function is given according to

$$ \rho_{mc}(\Gamma) = \frac{1}{\Omega}\delta\left(H(\Gamma) - E\right) $$

This means that there is an additional restriction in the probability density that it can only take non-zero value when the energy of the system is equal to E. How can I impose this constraint to match the $\rho_e(\Gamma)$ with $\rho_{mc}(\Gamma)$?

EDIT 1 (A possible approach) - Transformation of random variables

A possible approach is to consider the essential role of Liouville's theorem on equilibrium probability density, which suggests that the $\rho(\Gamma)$ depends on the phase-space explicitly only through the Hamiltonian ($H(\Gamma)$) of the system.

From the basis of probability theory, this can be considered as a transformation from the random variable $\Gamma$ to $E = H(\Gamma)$, such that the probability distribution, $\rho(\Gamma)$ and $\rho_{mc}(E)$ are according to,

$$ \rho_{mc}(E) = \int d\Gamma \rho(H(\Gamma) - E) $$

where we impose that the system has energy $E$ is imposed through the Dirac-delta function.

This approach particulalry emphasizes that the random variable of interest is not phase-space coordinates, but it's the total energy of the system (which is a constant of the motion)

Answer 1

In general, the two distributions have little to do with each other. After all, $\rho_e$ is fixed but $\rho_{mc}$ depends on the variable energy $E$. I will therefore reformulate your question as: can a suitable choice of $E$ allows you to recover $\rho_e$?

Small mistake in your normalisation of the microcanonical ensemble, you should rather write: $$ \rho_{mc} = \frac1W\delta(H-E) \\ W = \int\delta(H-E)d\Gamma $$ with $W$ essentially representing the area of the energy shell $H=E$, or equivalently the density of states in energy, which is different from $\Omega$ with and $E$ dependence.

To answer your question, it is best to use the canonical ensemble. It allows you to include the energy $E$ in the maximum entropy principle. You just add the constraint that the mean energy is fixed: $$ \int H\rho_cd\Gamma = E $$ This gives the usual: $$ \rho_c = e^{\beta F-\beta H} $$ The free energy $\beta F$ correspons to the normalisation constraint Lagrange multiplier: $$ Z = e^{-\beta F}=\int e^{-\beta H}d\Gamma $$ The inverse temperature $\beta$ corresponds to the energy constraint Lagrange multiplier.

In the thermodynamic limit, $\rho_{mc}$ and $\rho_c$ give the same statistics for macroscopic variables. You can now reformulate the original problem in terms of temperature. For which $\beta$ do you have $\rho_e=\rho_c$? From direct inspection, it is just infinite temperature, i.e. $\beta\to0$. Inverting the relationship between energy and temperature given either will give you the corresponding energy $E$. Explicitly, you can relate energy and temperature using the microcanonical ensemble: $$ \beta=\frac{\partial S}{\partial E} \quad S=\ln W $$ so the right energy $E$ corresponds to the one of maximum density of states or equivalently maximum entropy.

An important caveat is that this correspondence is exact for the canonical ensemble but for the microcanonical ensemble, it only applies to macroscopic variables in the thermodynamic limit. After all, there is no way that you can rigorously equate $\rho_e=\rho_{mc}$ for suitable $E$ unless $H$ is constant. This is highly unintuitive from a low dimension perspective, bit in the limit of infinite dimensions, the density of states is highly peaked around its maximum value so most of the uniform distribution’s support is on the maximal density energy shell.

In general, this is as far as you can go as it will depend on the density of states. Since your phase space is compact, you have a finite range of energy and the one of maximum density is “somewhere in the middle.”

A simple example for this is $N$ independent spins with an external magnetic field: $$ H=\sum_{i=1}^Ns_i^z $$ with $s\in (S^2)^N$. Energy ranges from $E\in[-N,N]$ and intuitively the maximum of density of states is at $E=0$ corresponding to infinite temperature. In fact $E<0$ is positive temperature and $E>0$ is negative temperature. You can check this quantitatively: $$ \beta F= -N\ln\left(\frac{\sinh\beta}\beta\right) \\ E=N\left(\frac1\beta-\coth\beta\right)\\ S=E-\beta F=N\left[\frac1\beta-\coth\beta+\ln\left(\frac{\sinh\beta}\beta\right)\right] $$

Answer 2

As an earlier related question correctly pointed out:

Jayne's introduced the effectiveness of Bayesian Inference in deriving different statistical ensembles. His idea was that the distribution that maximizes the Information Entropy of the system subject to certain physical constraints, lead to different ensembles in statistical mechanics

We deal here with applying different constraints - the confusion results from not specifying the region of integration in phase space, $\partial\Gamma$: $$ S[\rho(\Gamma)] = - \int_{\partial\Gamma} \rho(\Gamma) \ln (\rho(\Gamma)) d\Gamma $$

• if $\partial\Gamma$ is the surface defined by condition $H(\Gamma)=E$, then the derivation is correct, and predicts the equal probability of all the states - that is the microcanonical ensemble. So there is no contradiction.
• if $\partial\Gamma$ is the complete phase space and a constraint on energy is added: $$ \tilde{S}[\rho(\Gamma)] = S[\rho(\Gamma)] + \lambda \left( \int \rho(\Gamma) d\Gamma - 1 \right) + \beta \left( \int E(\Gamma)\rho(\Gamma) d\Gamma - E_0 \right), $$ then the Boltzmann/macrocanonical distribution results: $$ \rho(\Gamma)\propto e^{-\beta E(\Gamma)}. $$

To put it in different words: the confusion results from different meaning of constraint when talking about microcanonical ensemble (constraining the phase space to the states with certain energy) vs. the Jaynes' derivation (loking for a probability of states with certain energy in a much larger phase space.)