I have a bunch of doubts and confusions on the concept of entropy which have been bothering me for a while now. The most important ones are of a more technical nature, arisen from the reading of this and Jaynes' paper "Gibbs vs Boltzmann entropies". Although they seem like great texts, they leave me confused when they argue that the Gibbs entropy remains constant and that this is precisely what one needs to prove in the Second Law. I quote, from the first link:

we return to the specific case of a gas of N particles, this time confined to one side of a box containing a removable partition. We suppose that the initial state is such that we can descnbe it using the canonical probability distribution. From our earlier discussion we can then say that the Gibbs entropy SG is maximized and equal to the experimental entropy SE.

We now suppose that the partition is opened and the atoms occupy the whole box. We wait until the state variables stop changing, so in that sense the system is in equilibrium and a new experimental entropy S'E can be defined. Also, all the motions of the gas molecules are Hamiltonian, so that the Gibbs entropy S'G has not changed: S'G = SG

The probability distribution of the N particles is no longer the canonical one, however, because of the (very subtle!) correlations it contains reflecting the fact that the molecules were originally on one side of the partition. This means that the Gibbs entropy S'G is now in general less than the maximum attainable for the new values of the state variables, which is in turn equal to the new experimental entropy. So

SE = SG = S'G ≤ S'E

Well, I can say they totally lost me here. After the expansion, the system reaches an equilibrium. We shouldn't care about how we reached that equilibrium, so I would think that maximizing $S_G= -K_B \int \rho \log \rho d \mu$ -with $\rho$ the joint probability distribution of the position and momentum of the $N$ particles and with the restriction that the average energy is, say, $U$- should give again the probability distribution $\rho$ in the new equilibrium. So, I defintely don't understand at all why they say that "The probability distribution of the N particles is no longer the canonical one," neither do I understand the statement "because of the (very subtle!) correlations it contains reflecting the fact that the molecules were originally on one side of the partition," since I believe it is not important at all how we reached the equilibrium! If this was the case, why could we prove that in the first case (when all the molecules are in one side of the box) the Gibbs entropy coincided with the "experimental" one (I, by the way, don't know exactly what they mean by "experimental" entropy; do they mean Clausius'?). Didn't Jaynes prove that Gibbs entropy coincide with Clausius'? (I got that from his paper, at least). But how can Gibbs entropy remain constant if the entropy "MUST" increase?

Jaynes, in his paper, writes things like $(S_G)_{2}-(S_G)_{1},$ so that changes in the Gibbs entropy must be something meaninful, despite being so clear that the hamiltonian evolution leaves $\rho$ the same.

Well, I guess it's being really hard to explain accurately the nature of my confusions, but hopefully someone around here has struggled with similar issues and can give an enligthening clarification.

Thank you very much in advance.

EXPANDING THE ORIGINAL QUESTION: Let me follow the notation from Jaynes' paper, which I linked above. If a let the gas go through a free adiabatic expansion, since the evolution is hamiltonian, it is clear that $\frac{dW_N}{dt}$ obeys Liouville equation, but since $\{ e^{-\beta H}, H \}=0,$ it is clear that the $W_N$ remains constant and thus so do the Gibbs entropy $S_G.$ However, at the end of section IV in Jaynes' paper, he states

If the time-developed distribution function $W_N(t^{\prime}$...

And then I don't know what's going on anymore..! Does the $W_N$ change in time or not?

And, if in a thermal equilibrium we do not use the Canonical Ensemble, which ensemble are we using instead? How is the distribution function? What is the mathematical expresion of the new macroscopic restrictions we should add when maximizing the entropy functional to derive that new probability distribution?

  • 1
    $\begingroup$ I wrote an answer to a slightly different question here. It discusses some of the history of how entropy came about and provides some useful references on the topic (though I do not think it answers your question... but it might be helpful reference-wise). $\endgroup$ Commented Feb 8, 2016 at 14:33
  • $\begingroup$ great answer and thanks for the link to it. +1 on both $\endgroup$ Commented Feb 8, 2016 at 16:10
  • $\begingroup$ $W$ does change in time, since the expansion means Hamiltonian changes. What does not change is information entropy (the Gibbs entropy). Check my answer here: physics.stackexchange.com/questions/256302/… $\endgroup$ Commented May 17, 2016 at 21:09
  • $\begingroup$ See physics.stackexchange.com/a/288581/226902 $\endgroup$
    – Quillo
    Commented Jun 10 at 0:43

2 Answers 2


You say:

We shouldn't care about how we reached that equilibrium

In fact the entire point of the example is that we do. I shall try to explain why.

Jaynes and Gull both work in the framework of Bayesian inference (I can recommend the introductory text: http://www.amazon.co.uk/Data-Analysis-A-Bayesian-Tutorial/dp/0198568320. The title may seem unrelated to thermodynamics, but is actually at the heart of science itself). One important concept they use is the Principle of Maximum Entropy. It states essentially what you said in your question:

so I would think that maximizing $ S_G=−K_B∫\rho \log\rho\: \mathrm d\mu $ - with ρ the joint probability distribution of the position and momentum of the N particles and with the restriction that the average energy is, say, U - should give again the probability distribution ρ in the new equilibrium.

This yields the canonical ensemble. Now the assumption behind this is that there is no prior information that we have about the system. But in the case of the box we DO have prior information about the system, namely that the particles used to be on the right of the box. So the particles are in a different ensemble. This ensemble could probably be calculated, but it would be complicated and is not necessary for the sake of his argument.

About your questions regarding the "experimental entropy": What they mean by that is the entropy that is defined by:

$$ \Delta S_E = \int_\rm{reversible} \frac{\partial Q}{T} $$

So yes, I guess that is what people also call "Clausius' entropy". What Jaynes proved is that $ S_G $ and $ S_E $ (choosing the right constant offset for $ S_E $) coincide for the canonical ensemble. So in the argument above we have noted that the ensemble changes from canonical to something else. The new ensemble will have same $ S_G $ (since it was shown to be constant under the equations of motion) but $ S_E $ will have increased. This is then a restatement of the second law of thermodynamics (for systems starting in the canonical ensemble. I have struggled to find a prove for systems starting in other ensembles, would be happy if someone could link me.)

So that should also answer your question about how $ S_G $ staying constant is consistent with the second law.

  • $\begingroup$ You say that the statistical ensemble we use here is different, because we have prior information about the system. But do we ? In many introductory texbooks, we try to calculate the speed of a single particle after a few collisions. We can show that after a very short period of time (say 10 collisions) we completely lose the information about the speed or position of the particle. So is it reasonable to assume that we can track these "subtle" correlations ? Would it really make a measurable difference in the entropy ? $\endgroup$
    – Dimitri
    Commented Feb 8, 2016 at 14:46
  • $\begingroup$ I am not 100% sure which calculation you are talking about. In the example used in the paper they are explicitly talking about a quantum system. And the whole point of the paper is that it does make a difference. For the relation to classical systems the comment about diffusion in this paper (bayes.wustl.edu/etj/articles/cmystery.pdf) might help. $\endgroup$ Commented Feb 8, 2016 at 15:29
  • $\begingroup$ I was thinking of a classical calculation of the variation in angle $\delta \theta$ in the speed of a particle after a collision. So maybe my misunderstanding comes from the quantum nature of the system. But still, I find it hard to believe that the fact that the particles were on the same size at time $t_0$ can affect the thermodynamics a long time after that. Or maybe it comes from these Bayesian statistics you were referring to ? $\endgroup$
    – Dimitri
    Commented Feb 8, 2016 at 15:48
  • $\begingroup$ I share Dimitri's concern. Furthermore, I feel there is a contradiction in Jaynes' paper. Surely that's my lack of understanding, but I'll tell you what I feel by expanding my original question. Thanks! $\endgroup$
    – Qwertuy
    Commented Feb 8, 2016 at 15:48
  • 1
    $\begingroup$ The article you linked to is very interesting. The canonical ensemble point of view doesn't take into account the time-reversal symmetry breaking in the information we have about the system: we know its past, but not its future. It kind of makes reasonable why the canonical description fails to account for the experimentally measured entropy. Thanks ! $\endgroup$
    – Dimitri
    Commented Feb 8, 2016 at 16:34

There is great confusion about the Liouville equation and what it represents –or doesn't.

It represents a mechanical system that that undergoes mechanical processes that are strictly reversible, that is, processes that are isentropic.

It does not (in fact, it cannot) represent a thermodynamic system that exchanges heat, heat is not a concept that mechanics understands. It cannot represent the evolution of a system whose entropy changes. And it does not/cannot represent relaxation to equilibrium.

Many have argued, incorrectly, that since the Gibbs entropy, $-\int \rho(\Gamma)\log\rho(\Gamma)) d\Gamma$, remains unchanged in the Liouville equation, this must mean that the Gibbs entropy is wrong, or incomplete, or not exactly the Clausius entropy and so on. This argument blames Liouville's inadequacies on Gibbs.

The Liouville equation is inadequate because it applies deterministic mechanics, and as we know, thermodynamics requires statistical mechanics. The determinism that plagues Liouville arises from the fact that it treats the boundaries of the system as deterministic. As Gibbs put it:

The forces are supposed to be determined for every system by the same law, being functions of the coordinates of the system $q_1,\cdots q_n$, either alone or with the coordinates $a_1$, $a_2$, etc. of certain external bodies. It is not necessary that they should be derivable from a force-function. The external coordinates $a_1$, $a_2$, etc. may vary with the time, but at any given time have fixed values. In this they differ from the internal coordinates $q_1,\cdots q_n$, which at the same time have different values in the different systems considered. (Gibbs, Elementary Principles in Statistical Mechanics, p. 5).

The "external coordinates" represent interactions with the surroundings and are treated as deterministic functions of time. Of course the surroundings are made of particles whose motion depends on degrees of freedom that are not included in the Liouville equation. This is the approximation in the Liouville treatment that robs it of the ability to exhibit thermodynamic behavior.

To return to Jaynes's view, it is our ignorance about the precise state of the surroundings that increases entropy: a real system evolves differently from what Liouville predicts because we are ignorant of information that is necessary to predict the future state. We end up with more microstates than we thought we had at $t=0$, because we did not know the number of microstates available in the surroundings of the system. These microstates "seep" into the system in the form of entropy generation.

  • $\begingroup$ You're confusing mechanical reversibility with thermodynamic reversibility. System that obeys the first kind can also violate the second kind, e.g. expansion of hard sphere gas into larger container is reversible mechanically, and irreversible thermodynamically, and the Gibbs entropy functional (obeying the Liouville equation) is conserved, while experimental entropy implied by the gas macrostate increases (whether the macrostate is defined as coarse-grained microstate, or even the simplest set of macroscopic constraints like volume, energy, number of particles). $\endgroup$ Commented Feb 22, 2023 at 22:51
  • $\begingroup$ Thermodynamically irreversible processes happen also in isolated systems, where no heat or external influences are present, but some external intervention is made, such as removal of a wall. Jaynes explains how observation of processes in such system that do not decrease entropy is fully compatible with reversible Hamiltonian evolution, and if we assume the Maxent principle, how the latter explains why consistent non-decrease of experimental entropy follows from constancy of the Hamiltonian-defined evolution of the Gibbs entropy functional. $\endgroup$ Commented Feb 22, 2023 at 23:03
  • $\begingroup$ @JánLalinský You have misunderstood Jaynes as many have. Jaynes never suggested that the second law is compatible with reversible mechanics, because it isn't: there is no mechanically reversible path from one entropy contour to another because reversible mechanics is isentropic. What Jaynes explained is how to read the Liouville equation in light of the second law. He was defending Gibbs's entropy against those who argue that Liouville "proves" that Gibbs is wrong. $\endgroup$
    – Themis
    Commented Feb 23, 2023 at 23:33
  • $\begingroup$ My point, which you have also misunderstood, is that to make Liouville agree with Gibbs we must remove the assumption of determinism from Louisville's boundary conditions. $\endgroup$
    – Themis
    Commented Feb 23, 2023 at 23:33
  • $\begingroup$ There is no point claiming what Jaynes never suggested, you can't prove that such thing did not happen. His derivation of 2nd law from the Maxent principle and reproducible asymmetric relation of states A,B assumes also reversible Hamiltonian evolution of the phase density and thus 2nd law in that derivation is compatible with reversible Hamiltonian evolution. Reversible mechanics is isentropic in the sense the Gibbs functional is conserved, and this is compatible with 2nd law. If you don't agree with this, read Jaynes' paper Gibbs vs Boltzmann entropies. $\endgroup$ Commented Feb 24, 2023 at 1:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.