# Maxwell-Boltzmann Distribution (speed) as a Maximum Entropy Distribution and Its Interpretation

I am aware of the typical "physics" way of deriving the Maxwell-Boltzmann Distribution for speed $v$:

$p(v) = \sqrt{\left(\frac{m}{2\pi k_B T}\right)^3} 4\pi v^2 \exp\left(-\frac{mv^2}{2k_BT}\right)$

from isotropy of $\vec{v}$, integrate over directions to get a function of $v$ only, partition function, and whatnot.

I also know that Maxwell-Boltzmann Distribution can arise as a result of maximizing (differential Shannon) Entropy $\mathcal{H} = \int -p \ln{p} \, \mathrm{d}v$ under certain constraints. In particular, the constraints are on $E(v^2)$ and $E(\ln(v))$. (https://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution#Other_examples)

I see why this 'works' mathematically.

However, my question is: Is there some "physical" interpretation of the second constraint, i.e. on the expected value of $\ln(v)$? The first constraint is merely that of expected kinetic energy to be some constant (its physical significance being related to temperature), but I have no idea what the $\ln(v)$ implies.

• I think its a very weird way of writing the constraint that probabilities must sum to 1. I can't see an obvious way of showing this, however, without starting with the Boltzmann distribution. In any case the normal constraints given are that probabilities sum to 1 and the mean energy is fixed. – By Symmetry Jul 3 '17 at 12:09
• @BySymmetry No, it is not the normalization constraint. The normalization constraint is merely written as $\int_0^\infty p(v) \, \mathrm{d}v = 1$, and this with a Lagrange multiplier merely adjusts the constant scale factor up front. The $\ln{v}$ constraint adds the $v^2$ dependency outside of the exponential, when you use a Lagrange multiplier to maximize $\mathcal{H}$. – Sanha Cheong Jul 3 '17 at 12:15
• If you only use the normalization & the mean constraints on non-negative real valued $v$, one gets an exponential distribution $p(v) = \lambda \exp \left(-\lambda x\right)$ (mean = $1/\lambda$) as the MaxEnt distribution. – Sanha Cheong Jul 3 '17 at 12:17
• In that case is sounds like it is determining the velocity dependence of the kinetic energy. – By Symmetry Jul 3 '17 at 12:25
• @BySymmetry I don't think that's the case either. As written in the original question, the constraint $E(v^2)$ is the mean energy constraint. It is a constraint on not the mean speed but the mean of speed squared (i.e. the kinetic energy, up to a constant). – Sanha Cheong Jul 3 '17 at 12:30

## 1 Answer

Short Answer.

Including the constraint on the expectation of the log of the speed is equivalent to assuming a uniform quantization of classical phase space which, from experience, we know is the correct prescription for applying MaxEnt to classical statistical mechanics.

Details.

Jaynes showed that the differential entropy is only an appropriate continuum generalization of the discrete Shannon entropy if the discretization one chooses is uniform. For a nice discussion of this, I'd recommend first reading section 4b. of Jaynes's Information Theory and Statistical Mechanics lectures since that seems like the original source if this observation. I also found a nice Wikipedia article discussing this point:

Limiting density of discrete points

Jaynes showed that if one wants to generalize the information entropy from a probability mass function on a finite state space to a continuous probability distribution on some subpset of $\mathbb R^n$, like when we deal with classical phase space, then although the differential entropy you wrote down seems like the obvious generalization one would obtain from discretizing the space and applying the discrete version of the entropy, it actually implicitly assumes a uniform discretization, like splitting up the space into cubes of equal volume. The more general expression that allows for a discretization in which the density of states $m(\vec x)$ on the space is not necessarily uniform is $$H[\rho]=-\int \rho(\vec x) \ln \frac{\rho(\vec x)}{m(\vec x)}\, d^n \vec x.$$ Jaynes concedes that in the context of classical mechanics it's not clear what justifies choosing a particular $m$ over another, but he argues that if, motivated by quantum mechanics, we use the usual "quantization" trick of splitting phase space into cells of equal volume $d^{3}pd^{3}q/h^{3}$, then we should correspondingly choose $m(\vec x) = const.$. If we do this and drop an unimportant overall constant that results, we can use the naive differential entropy expression $$H[\rho] = \int\rho(\vec p, \vec q)\ln \rho(\vec p, \vec q)\, d^{3}\vec pd^{3}\vec q$$ On the other hand, notice that if you want to describe the statistics of a classical system using a probability distribution on the space of momentum magnitude $p = |\vec p|$ (or equivalently speed), then the uniform density of states in momentum space turns into an increasing density of states in $p$-space that depends on the dimension of the space To see this, note that uniform density of states in phase space corresponds to the fact that the number of states in a given region is proportional to its volume. In $d$-dimensions, the number of states with momenta between $p$ and $p+dp$ is therefore proportional to $p^{d-1}\, dp$. It follows that we should take $$m(p) = (\text{const.})\,p^{d-1}$$ Ignoring unimportant overall additive constants resulting from the normalization of $m$ and then including $m\sim p^{d-1}$ in the computation of $H$ is equivalent to adding a Lagrange multiplier term in which the Lagrange multiplier has value $d-1$. To see this, notice on one hand that including $m$ and using the more general version of $H$ causes us to find critical points of the following functional: \begin{align} J[\rho] &= -\int \rho(p)\ln \rho(p)\, dp \\ &\hspace{1cm}+ \lambda_0\left(\int \rho(p)\, dp - 1\right)+ \lambda_1\left(\int \rho(p) p^2\, dp - C\right) + (d-1)\int \rho(p)\ln(p)\, dp \end{align} On the other hand, using the differential entropy and adding a Lagrange multiplier term causes us to find critical points of the functional \begin{align} G[\rho] &= -\int \rho(p)\ln \rho(p)\, dp \\ &\hspace{1cm}+ \lambda_0\left(\int \rho(p)\, dp - 1\right)+ \lambda_1\left(\int \rho(p) p^2\, dp - C\right) + \lambda_2\left(\int \rho(p)\ln(p)\, dp - K\right) \end{align} When we find critical points of $G$, we find that $$\rho(p) = C e^{\lambda_1 p^2}e^{\lambda_2 \ln p} = C p^{\lambda_2 }e^{\lambda_1 p^2}$$ If we appropriately choose $K$ so that $\lambda_2 = d-1$, then we obtain $$\rho(p) = C p^{d-1}e^{\lambda_1 p^2}$$ For $d=3$ we obtain a $p^2$ factor in $\rho$, which is precisely what we have for the $3$-dimensional Maxwell speed distribution.

• Thank you very much for this detailed explanation. Could you please elaborate how these considerations can be expressed as a constraint on $<\log v>$, and in particular where the constraint value listed on the Wikipedia page comes from? – user8153 Jul 3 '17 at 20:58
• @user8153 I'd recommend reading the following which discusses your questions in a general setting: en.wikipedia.org/wiki/… I haven't myself worked out the algebraic details of the relationships between the Lagrange multipliers and the expressions constraining the expectation values in this particular instance, but f I have time/energy to post an addendum in this regard, I will. – joshphysics Jul 3 '17 at 21:14
• Thank you! I figured roughly that's where it was coming from, but this certainly helps. @user8153 The answer to the second question comes from the fact that in 3-dimensional space, the "degeneracy" of $v$ is over a spherical shell, whose volume goes as $v^{n-1}(2\pi)^{n/2}/\Gamma(n/2)\mathrm{d}v$. The Gamma function term in the denominator leads to the "not-so-obvious" appearance of the Euler Gamma constant. – Sanha Cheong Jul 4 '17 at 0:42
• Is this just the same as performing maxent with a $v^2$ constraint in 3 dimensions (giving $e^{\lambda v^2}$, then switching to radial coordinates and integrating over the angles? The latter should also result in a factor $v^2$ from change of variables – innisfree Jul 27 '17 at 11:54