# Derive statistical entropy from information entropy

I'm trying to compute the entropy of an Hamiltonian system with a fixed energy from the information entropy.

Consider a random variable $X$ (state) to be uniformly distributed in a domain $\Omega$ (phase-space), $$P(x) = 1/|\Omega|$$, with $|\Omega| = \int_\Omega dx$.

Consider a second random variable (energy) defined by $Y=H(X)$, such that $P(y|x) = \delta(y - H(x))$.

The marginal of $y$ is $$P(y) = \int_\Omega P(y|x)P(x) dx = \frac{1}{|\Omega|}\int_\Omega \delta(y - H(x)) dx$$

which is the density of states. Moreover, the entropy is given by the entropy of $X$, which is given by $$S(X) = -\int_\Omega P(x) \log P(x) dx = \log |\Omega|$$

My question is how to derive the entropy of the system with a fixed energy. For that, I'm computing the entropy of the system conditioned that it has a given energy $y$, $P(X|y)$.

1. Is this correct?

I would expect it to be $S(X|y) = \log |\Omega(y)|$ where $|\Omega(y)| = |\Omega|P(y) = \int_\Omega \delta(y - H(x))dx$ is the volume of the phase-space for a given energy $y$, but I'm not being able to get it.

Let's try to do the calculation:

$$S(X|y) = -\int P(x|y) \log P(x|y) dx$$

Using $P(X|Y) = P(Y|X)P(X)/P(Y)$ and $P(X) = 1/|\Omega|$, computing $S(X|y)$ gives

$$\log |\Omega|P(E) -\frac{1}{|\Omega| P(E)}\int_\Omega \delta(y-H(x)) \log \delta(y-H(x)) dx$$

The first term is what I would expect. But the second term I'm not being able to show that it is 0.

1. What am I doing wrong?

(I'm aware that this is not the traditional notation in Physics, but I'm on purpose trying to derive the result using a statistics formalism).

• The last line is the entropy of a ensemble with only one state occupied, so from a physical point of view: thats zero (and then everythings fine). However, I cant show that right now in a proper way
– Bort
Commented Dec 16, 2015 at 11:02
• Okay I for one am giving up, I take the physicist's way out: You cannot study the microcanonical ensemble with a fixed energy (because it isnt properly defined as you see), but you should consider to be in a small shell (between $y$ and $y+\Delta E$) you will find that anything works properly then
– Bort
Commented Dec 16, 2015 at 11:11
• Thanks for the try. I've improved the last equation to the result I have so far, which is closer to the expected result. Commented Dec 16, 2015 at 13:03
• If you did it the way with an energy shell the second term would be proportional to Delte E and you would therefore ignore it (as it does not contribute to the thermodyn. limit) and be happy
– Bort
Commented Dec 16, 2015 at 18:40
• I should add the technical reason for your problems |Omega(y)| is zero (youre looking at N-1 dimensional subset of N-dimensional space)
– Bort
Commented Dec 16, 2015 at 18:55