Entropy is defined as the sum of the probabilities of microstates times the logarithm.

$$ S = - k_B\sum_i p_i \ln p_i $$

The mean value of something is defined as

$$ \bar{x} = \sum_i x_i p_i $$

but for continuous cases is:

$$ \bar{x} = \int^{\infty}_{-\infty} x \rho_x \, \mathrm{d} x $$

Would the entropy for an infinitely fine system be the following?

$$ S = -k_B \int^{\infty}_{-\infty} \rho_x \ln \rho_x \, \mathrm{d} x $$

I realise that due to quantum mechanics and atomic composition of materials that stuff is discrete and not continuous.


The expression you wrote down is the so-called differential entropy of the system. In many cases (see below for an example), this is a useful expression, but there are some caveats.

First note that the differential entropy can be negative, because $p(x)$ can become greater than one. An easy example (which I took from the Wikipedia article ;)) is the uniform distribution $p(x) \equiv 2$ on the interval $\left[0,\frac 1 2\right]$: The differential entropy is obviously $-k_B \ln 2$.

More generally we find that the value of such a differential entropy does not have a physical significance, but only differences between values. For example, the Nernst Theorem ($\lim_{T \to 0} S = k_B \ln g_0$ where $g_0$ is the degeneracy of the ground state) will typically not hold for a differential entropy.

Why this is so you can understand by considering differential entropy as a discrete entropy of points, in the limit where the points are so close together that we can describe them as if we had a continuous system. (This is called a Limiting Density of Discrete Points.) As you can see in the Wikipedia article, the entropy $S_N$ of such a system differs from the "naive" differential entropy $s$ by $$ S_N = s + \ln\left( N / r \right) $$ where $N$ is the number of points and $r$ the size of the system (assuming the point density is constant). When we consider the differential entropy, we essentially neglect the "infinite but constant" term $\ln\left( N/r \right)$.

Finally, I promised you an example where this is used. If you consider a cavity, you can describe its density matrix using the Husimi Q function $\mathcal Q(q,p)$ which is a probability density. The Wehrl entropy of the system is $$ S_W = -k_B \int \mathcal Q(q,p) \ln \mathcal Q(q,p) \frac{\mathrm dq\, \mathrm dp}{2\pi\hbar}.$$ This is an interesting quantity which I wrote my bachelor's thesis about ;) people use it e.g. in entanglement theory and to describe decoherence in quantum optics.
A pretty cool fact is $S_W \geq 1$ which is a refinement of the Heisenberg uncertainty relation for the system. This so-called Lieb conjecture was proven only recently.


Yes, this would be the entropy for a continues system. Note there are also statistical treatments of classical theories. In those, entropy is exacly defined the way you wrote it there. $$ S[\rho]= - \int d\Gamma \rho(\Gamma)\log{\rho(\Gamma)} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.