# Deriving the ideal gas law from relative entropy, instead of differential entropy, to avoid negative entropy

Deriving the ideal gas law is one of the first examples presented in introductory statistical physics. It is derived using the differential entropy. Let $$\chi$$ be an uncountable set. Then the differential entropy is defined as

$$S=-k_B \int_\chi p(x) \ln(p(x))dx \, .$$

This entropy is known to admit negative entropy solution. In the case of the ideal gas law, derived using the differential entropy, the entropy $$S$$ is (reference)

$$S=\nu R \left( \ln[V]+\frac{3}{2}\ln[T]+\frac{3}{2}\ln \left[ \frac{2\pi m}{h_0^2} \right] +\frac{3}{2} \right)$$

which according to the author, $$S\to-\infty$$ if $$T\to 0$$.

An alternative definition for entropy exists which admits no negative solutions for the entropy. This is the relative entropy and it is defined as

$$D[P||Q]=\int_{-\infty}^{\infty} p[x] \ln \left[ \frac{p[x]}{q[x]} \right]dx \, .$$

Can we derive the ideal gas law using the relative entropy as the starting point, such that the entropy of an ideal gas is positive for all values of $$T$$?

• The log function cannot be applied to dimensionful quantities like volume ($V$) and temperature ($T$). There's something fundamentally wrong with the equation from the reference. – DanielSank Mar 3 '19 at 22:42

I don't believe that this can be done, if you insist on applying the ideal gas model all the way down to absolute zero. As you can see on the Ideal Gas Wikipedia page the logarithmic dependence on $$T$$ (at constant $$N$$ and $$V$$, let's say), which becomes troublesome as $$T\rightarrow 0$$, is a consequence of classical thermodynamics, not from a particular statistical mechanical definition of entropy. Whatever definition you attempt to start from, the result must still agree with classical thermodynamics, up to some arbitrary constant which the classical approach may not be able to pin down. This will not remove the divergence. The problem can only be resolved by accepting that the ideal gas model becomes inapplicable at some point before we reach absolute zero.

• It'd probably be a bit more accurate to say that the logarithmic dependence on $T$ at low T is a consequence of classical thermodynamics. The high-$T$ dependence is still logarithmic, and that limit works just fine for quantum-mechanical "ideal gases". Still, +1. – Michael Seifert Mar 3 '19 at 23:32
• Thanks @MichaelSeifert I've amended my answer to (hopefully) make that more clear. – user197851 Mar 4 '19 at 8:56
• @MichaelSeifert what do you call classical thermodynamics exactly? – gatsu Mar 7 '19 at 16:52
• @gatsu: As a rough definition, I'd describe "classical thermodynamics" as situations where the underlying microstates are counted via a volume on some classical phase space, rather than via a set of discrete states. The "$S \to - \infty$" problem arises in classical thermodynamics arise when the accessible region of phase space goes to zero. – Michael Seifert Mar 7 '19 at 16:59

The pathological $$\sim \ln T$$ dependence of the entropy of an ideal gas also occurs in axiomatic thermodynamics (i.e. 19th century theory of thermodynamics devoid of statistical mechanics considerations) given the ideal gas law $$PV=nRT$$. So, as far as my understanding is concerned, the problem is not rooted in statistical mechanics (classical or otherwise) but in the model used for the equation state.

In thermodynamics the problem can be traced back to the fact that the pressure $$P$$ of the ideal gas law is proportional to the absolute temperature $$T$$ for all temperature values. But since thermodynamics does not provide any recipe to derive or predict equations of state, this is not an issue: the ideal gas law is a model that works fine for a range of temperatures for most gases.

The thermodynamics practitioner will just look for empirical results on which to base low temperature equations of state. After all, not only is it expected that a gas at fixed pressure eventually becomes liquid (which is not captured by the ideal gas law) but it is also expected that for most substances the liquid becomes solid too below some threshold temperature. There is therefore nothing abnormal in the fact that the ideal gas law doesn't work fine down to absolute zero.

Thus, if the goal of statistical mechanics is to be compatible with thermodynamics or even to retrieve it, then the ideal gas entropy derived from N classical point particles in a box is all fine (since it coincides with thermodynamics would tell us of an ideal gas).

Now, the question you ask about non-negative entropies is not limited to an ideal gas model but is a feature and problem of the differential Shannon entropy. Some have called it the measure problem and proposed some way forward about it.