# Do information entropy and thermodynamic entropy evolve the same way?

Recently I asked a question about the relationship between Von Neumann entropy and heat transfer. The link is here.

Is the Von Neumann entropy related to heat transfer?

I feel as though the answer is not quite so clear so I will ask a couple of questions that I think may essentially be the same question as this previous one or, if these new questions aren't the same as the old question, your answers will help me clarify some key points.

Original question: Is the change in Von Neumann entropy related to heat transfer? I think I can rephrase and ask if information theory can explain heat transfer in terms of evolving information.

New Question #1:

The Wikipedia page "Maximum entropy thermodynamics" says under "Second Law": (paraphrased) Liouville's equation shows how the phase space evolves from a maximum entropy distribution to one that is not necessarily maximum entropy, yet the thermodynamic entropy is certainly maximized.

Does this mean what I think it does: that the evolution of information entropy and thermodynamic entropy are not the same? When I see people comparing information and thermodynamic entropy they always do it for a single state. But I don't think I've ever seen a discussion of whether, if the state evolves, the two entropies evolve to the same value. The Wiki article suggests to me that they don't. So, if they don't behave exactly the same, aren't they different things?

New Question #2:

Another Wiki article "Entropy in thermodynamics" says, under "Criticisms": (paraphrased) information has no concept of temperature or energy so thermodynamic and information entropies are not the same. Does anybody disagree with this?

• If your out of equilibrium, defining temp is hard so thermo dynamic entropy isn’t well define. Also, I don’t think anyone disagrees. I have to think more to give a better answer – Shane P Kelly Feb 14 '18 at 1:14
• This doesn't have to be about non-equilibrium. If the state evolves from one equilibrium to the next, is the information entropy of the new equilibrium state the same as the thermodynamic entropy of that new equilibrium state? – SuchDoge Feb 26 '18 at 18:29
• @SuchDoge: Your main question seems to be: “Is von-Neumann entropy equivalent to the thermodynamic entropy in a physical sense?” Is my impression correct? – AlQuemist Mar 14 '18 at 9:04
• "Liouville's equation shows how the phase space evolves from a maximum entropy distribution to one that is not necessarily maximum entropy, yet the thermodynamic entropy is certainly maximized. " I wasn't able to find this in the wiki article. The section on second law is discussing an issue with Liouville's theorem and how it requires you to take averages over time to account for the increase in entropy. I'll explain more in my answer – Shane P Kelly Mar 14 '18 at 14:05
• Closely related, if not a duplicate: physics.stackexchange.com/questions/263197/… – Rococo Mar 18 '18 at 18:32

Question 1

Liouville's equation shows how the phase space evolves from a maximum entropy distribution to one that is not necessarily maximum entropy [...]

I cannot find a reference to Liouville's equation in Wikipedia's entry on the second law. In fact, the above statement is wrong: Liouville's equation preserves entropy at all times, that is, the Liouville equation is capable of describing only an isentropic process.

Question 2

Another Wiki article "Entropy in thermodynamics" says, under "Criticisms": (paraphrased) information has no concept of temperature or energy so thermodynamic and information entropies are not the same. Does anybody disagree with this?

The criticism is unfounded on several levels:

• Shannon only claimed to have identified the entropy functional. He was not interested in "energy" or "temperature". That doesn't mean that these concepts cannot exist within the formalism of information theory.
• "Temperature" and "energy" are just names of variables that appear in certain physical problems. Jaynes realized this and drew the connection between Information Theory and Statistical Mechanics. Briefly, any positive variable $$x$$ whose mean is known is "energy"; "temperature" arises as a Lagrange multiplier when one seeks the probability distribution of $$x$$ that maximizes entropy given that its mean is known.
• It is possible to recover the entire calculus of thermodynamics starting with the notion of the most probable distribution (https://arxiv.org/abs/1809.07367).