# The statistical physics of a simple continuous system

Let $$\mathbb{R}_{\geq 0}$$ be the set of real numbers greater or equal to zero. Assume an average value $$\overline{R} \in \mathbb{R}$$, called the prior. Then, the probability distribution $$p(r), \forall r \in \mathbb{R}_{\geq 0}$$ which maximizes the entropy based on the prior is the Gibbs distribution. Since $$\mathbb{R}$$ is an uncountable set, the partition function is an integral:

$$Z=\int_0^\infty e^{-\beta r}dr=\frac{1}{\beta}$$

The average value $$\overline{R}$$ is

$$\overline{R}=-\frac{1}{Z}\frac{\partial Z}{\partial \beta}=-\beta (- \beta^{-2})=\frac{1}{\beta}$$

and the entropy is

$$S=k_B(\ln [Z]+\beta \overline{R})=k_B\left(\ln[\beta^{-1}]+\beta \frac{1}{\beta}\right)=k_B \left( 1-\ln [\beta] \right)$$

Graphing the entropy yields:

Why is the entropy negative when $$1<\ln[\beta]$$? Should the entropy not be greater than zero for all values of $$\beta$$? Error somewhere?

EDIT:

As requested in the comments, the Log Plot of S is

and $$\beta$$ is a Lagrange multiplier.

EDIT-2:

As clarified in the comments, here is the plot of $$S$$ with respect to $$\ln{\beta}$$.

• Two things: 1) please define $\beta$, and 2) plot the entropy on a log scale. It's weird that you've defined the average to be $\bar{R}$ but then introduce $\beta$ as a separate parameter. Mar 2, 2019 at 19:51
• @DanielSank I'm assuming $\beta=1/kT$ , at least this is what I have always seen. Mar 2, 2019 at 20:00
• Alexandre, are you looking at a system whose energy varies linearly with respect to some parameter $r$, and where there is no degeneracy for any energies? Mar 2, 2019 at 20:02
• Please make your question one cohesive post. An edit history is available for those who are interested. Mar 2, 2019 at 20:08
• @Aaron correct, it physically corresponds to a system where the energy varies linearly and where there is no degeneracy. One way to think of it is an entropic force $F$ over a distance $x$. In this case $E=Fx$. Mar 2, 2019 at 20:13

## 1 Answer

Look up differential entropy.

For continuous distributions, you have to compute the differential entropy, which is expected to be negative sometimes, and not the Boltzmann entropy or the Shannon entropy or the von Neumann entropy.

The mutual information is always positive, even the mutual information between two continuous distributions where the differential entropy is negative.

• That answer is concerning. Since most of introduction statistical physics is based on the differential entropy, how deep does the error go? For instance, the idea gas law is obtained using the differential entropy. Do both definitions accidentally agree on physically-relevant cases (it seems they would not)? Mar 2, 2019 at 22:59
• Differential entropy and Boltzmann entropy more or less agree, up to an additive normalizing constant. The problem is that, with continuous distributions, there is really no such thing as a microstate (like Boltzmann entropy requires). For any coarse-grained definition of microstates, you can always pick a finer-grained definition one. This is why you end up with differential entropy. Mar 3, 2019 at 3:51
• It's perfectly possible to make introductory statistical physics perfectly rigorous, using the techniques that mathematicians and information theorists use to deal with differential entropy. But physicists tend to just sweep these minor inconsistencies under the rug and pretend entropy is always positive, probably because they're not worried about proving theorems but about making their theories agree with experiment. Mar 3, 2019 at 3:53
• do you believe it is possible to derive the ideal gas law starting with relative entropy? Mar 3, 2019 at 15:00
• @AlexandreH.Tremblay Yes it is, and if you'd like to post a question asking how and link it here, I will be happy to answer. Mar 3, 2019 at 15:04