I have a bunch of doubts and confusions on the concept of entropy which have been bothering me for a while now. The most important ones are of a more technical nature, arisen from the reading of this and Jaynes' paper "Gibbs vs Boltzmann entropies". Although they seem like great texts, they leave me confused when they argue that the Gibbs entropy remains constant and that this is precisely what one needs to prove in the Second Law. I quote, from the first link:

we return to the specific case of a gas of N particles, this time confined to one side of a box containing a removable partition. We suppose that the initial state is such that we can descnbe it using the canonical probability distribution. From our earlier discussion we can then say that the Gibbs entropy SG is maximized and equal to the experimental entropy SE.

We now suppose that the partition is opened and the atoms occupy the whole box. We wait until the state variables stop changing, so in that sense the system is in equilibrium and a new experimental entropy S'E can be defined. Also, all the motions of the gas molecules are Hamiltonian, so that the Gibbs entropy S'G has not changed: S'G = SG

The probability distribution of the N particles is no longer the canonical one, however, because of the (very subtle!) correlations it contains reflecting the fact that the molecules were originally on one side of the partition. This means that the Gibbs entropy S'G is now in general less than the maximum attainable for the new values of the state variables, which is in turn equal to the new experimental entropy. So

SE = SG = S'G ≤ S'E

Well, I can say they totally lost me here. After the expansion, the system reaches an equilibrium. We shouldn't care about how we reached that equilibrium, so I would think that maximizing $S_G= -K_B \int \rho \log \rho d \mu$ -with $\rho$ the joint probability distribution of the position and momentum of the $N$ particles and with the restriction that the average energy is, say, $U$- should give again the probability distribution $\rho$ in the new equilibrium. So, I defintely don't understand at all why they say that "The probability distribution of the N particles is no longer the canonical one," neither do I understand the statement "because of the (very subtle!) correlations it contains reflecting the fact that the molecules were originally on one side of the partition," since I believe it is not important at all how we reached the equilibrium! If this was the case, why could we prove that in the first case (when all the molecules are in one side of the box) the Gibbs entropy coincided with the "experimental" one (I, by the way, don't know exactly what they mean by "experimental" entropy; do they mean Clausius'?). Didn't Jaynes prove that Gibbs entropy coincide with Clausius'? (I got that from his paper, at least). But how can Gibbs entropy remain constant if the entropy "MUST" increase?

Jaynes, in his paper, writes things like $(S_G)_{2}-(S_G)_{1},$ so that changes in the Gibbs entropy must be something meaninful, despite being so clear that the hamiltonian evolution leaves $\rho$ the same.

Well, I guess it's being really hard to explain accurately the nature of my confusions, but hopefully someone around here has struggled with similar issues and can give an enligthening clarification.

Thank you very much in advance.

EXPANDING THE ORIGINAL QUESTION: Let me follow the notation from Jaynes' paper, which I linked above. If a let the gas go through a free adiabatic expansion, since the evolution is hamiltonian, it is clear that $\frac{dW_N}{dt}$ obeys Liouville equation, but since $\{ e^{-\beta H}, H \}=0,$ it is clear that the $W_N$ remains constant and thus so do the Gibbs entropy $S_G.$ However, at the end of section IV in Jaynes' paper, he states

If the time-developed distribution function $W_N(t^{\prime}$...

And then I don't know what's going on anymore..! Does the $W_N$ change in time or not?

And, if in a thermal equilibrium we do not use the Canonical Ensemble, which ensemble are we using instead? How is the distribution function? What is the mathematical expresion of the new macroscopic restrictions we should add when maximizing the entropy functional to derive that new probability distribution?

  • 1
    $\begingroup$ I wrote an answer to a slightly different question here. It discusses some of the history of how entropy came about and provides some useful references on the topic (though I do not think it answers your question... but it might be helpful reference-wise). $\endgroup$ Feb 8, 2016 at 14:33
  • $\begingroup$ great answer and thanks for the link to it. +1 on both $\endgroup$ Feb 8, 2016 at 16:10
  • $\begingroup$ $W$ does change in time, since the expansion means Hamiltonian changes. What does not change is information entropy (the Gibbs entropy). Check my answer here: physics.stackexchange.com/questions/256302/… $\endgroup$ May 17, 2016 at 21:09

1 Answer 1


You say:

We shouldn't care about how we reached that equilibrium

In fact the entire point of the example is that we do. I shall try to explain why.

Jaynes and Gull both work in the framework of Bayesian inference (I can recommend the introductory text: http://www.amazon.co.uk/Data-Analysis-A-Bayesian-Tutorial/dp/0198568320. The title may seem unrelated to thermodynamics, but is actually at the heart of science itself). One important concept they use is the Principle of Maximum Entropy. It states essentially what you said in your question:

so I would think that maximizing $ S_G=−K_B∫\rho \log\rho\: \mathrm d\mu $ - with ρ the joint probability distribution of the position and momentum of the N particles and with the restriction that the average energy is, say, U - should give again the probability distribution ρ in the new equilibrium.

This yields the canonical ensemble. Now the assumption behind this is that there is no prior information that we have about the system. But in the case of the box we DO have prior information about the system, namely that the particles used to be on the right of the box. So the particles are in a different ensemble. This ensemble could probably be calculated, but it would be complicated and is not necessary for the sake of his argument.

About your questions regarding the "experimental entropy": What they mean by that is the entropy that is defined by:

$$ \Delta S_E = \int_\rm{reversible} \frac{\partial Q}{T} $$

So yes, I guess that is what people also call "Clausius' entropy". What Jaynes proved is that $ S_G $ and $ S_E $ (choosing the right constant offset for $ S_E $) coincide for the canonical ensemble. So in the argument above we have noted that the ensemble changes from canonical to something else. The new ensemble will have same $ S_G $ (since it was shown to be constant under the equations of motion) but $ S_E $ will have increased. This is then a restatement of the second law of thermodynamics (for systems starting in the canonical ensemble. I have struggled to find a prove for systems starting in other ensembles, would be happy if someone could link me.)

So that should also answer your question about how $ S_G $ staying constant is consistent with the second law.

  • $\begingroup$ You say that the statistical ensemble we use here is different, because we have prior information about the system. But do we ? In many introductory texbooks, we try to calculate the speed of a single particle after a few collisions. We can show that after a very short period of time (say 10 collisions) we completely lose the information about the speed or position of the particle. So is it reasonable to assume that we can track these "subtle" correlations ? Would it really make a measurable difference in the entropy ? $\endgroup$
    – Dimitri
    Feb 8, 2016 at 14:46
  • $\begingroup$ I am not 100% sure which calculation you are talking about. In the example used in the paper they are explicitly talking about a quantum system. And the whole point of the paper is that it does make a difference. For the relation to classical systems the comment about diffusion in this paper (bayes.wustl.edu/etj/articles/cmystery.pdf) might help. $\endgroup$ Feb 8, 2016 at 15:29
  • $\begingroup$ I was thinking of a classical calculation of the variation in angle $\delta \theta$ in the speed of a particle after a collision. So maybe my misunderstanding comes from the quantum nature of the system. But still, I find it hard to believe that the fact that the particles were on the same size at time $t_0$ can affect the thermodynamics a long time after that. Or maybe it comes from these Bayesian statistics you were referring to ? $\endgroup$
    – Dimitri
    Feb 8, 2016 at 15:48
  • $\begingroup$ I share Dimitri's concern. Furthermore, I feel there is a contradiction in Jaynes' paper. Surely that's my lack of understanding, but I'll tell you what I feel by expanding my original question. Thanks! $\endgroup$
    – Qwertuy
    Feb 8, 2016 at 15:48
  • 1
    $\begingroup$ The article you linked to is very interesting. The canonical ensemble point of view doesn't take into account the time-reversal symmetry breaking in the information we have about the system: we know its past, but not its future. It kind of makes reasonable why the canonical description fails to account for the experimentally measured entropy. Thanks ! $\endgroup$
    – Dimitri
    Feb 8, 2016 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.