From what I gather (and please correct me if I'm wrong), Jaynes argues that thermodynamic and information entropy are the same since the assumption in statistical thermodynamics that the energy distribution attained is that which maximizes the ways energy is distributed is equivalent to assuming the maximum ignorance distribution, which appears to me to be a subjective concept or at least one relating to information.

However, isn't this similarity just a coincidence? There are physical reasons energy maximizes the ways it is distributed (2nd law) having nothing to do with the fact that the distribution it attains happens to maximize our inability to describe the energy (maximizes ignorance). .

(Here I am talking about classical physics. I realize the uncertainty principle may unite information and thermodynamics in quantum.)

Update: The discussion in the link below has some claim that information and thermodynamic entropy are independent specific examples of a more general concept, but they are not equivalent.

Is information entropy the same as thermodynamic entropy?

  • $\begingroup$ Related: physics.stackexchange.com/q/398883/109928 $\endgroup$ – Stéphane Rollandin Apr 10 '18 at 19:02
  • $\begingroup$ We often use anthropomorphic terminology, e.g. "ignorance", when discussing mathematical or physical concepts. But that does not mean that these concepts are subjective. It simply means that we are appealing to our intuition in order to understand an abstract concept. $\endgroup$ – Lee Mosher Apr 11 '18 at 20:17
  • $\begingroup$ What paper and section do you refer to? I do not think Jaynes meant, generally, that thermodynamic entropy is the same as information entropy. I think he meant that thermodynamic entropy can be calculated/understood as information entropy of probability distribution, if that distribution is assigned based on macroscopic state of the system. $\endgroup$ – Ján Lalinský Apr 11 '18 at 20:47
  • $\begingroup$ @Jan Lalinsky, His "The Physical Review, Vol. 106, No. 4, 620-630" titles "Information Theory and Statistical Mechanics" in the introduction he states : "The mere fact that $-\sum p_i \log p_i$ occurs both in statistical mechanics and in information theory does not in itself establish any connection between these fields. This can be done only by finding new viewpoints from which thermodynamic entropy and information-theory entropy appear as the same concept. In this paper we suggest a reinterpretation of statistical mechanics which accomplishes this". $\endgroup$ – SuchDoge Apr 13 '18 at 19:10
  • $\begingroup$ He also says in that same article "...however, we can take entropy as our starting concept, and the fact that a probability distribution maximizes the entropy subject to certain constraints becomes the essential fact which justifies use of that distribution for inference". This is what I am arguing is not justified. Just because two ideas lead to the same result doesn't mean the ideas are the same thing. $\endgroup$ – SuchDoge Apr 13 '18 at 19:17

We needn't go to QM for the moment. If you take a lecture course in classical thermodynamics, the second law is derived from informational considerations, although they typically discuss it in terms of maximising the number (typically denoted $W$ or $\Omega$) of microstates per macrostate. We can then show the second law's entropy-increasing formulation is equivalent to heat moving from warmer sources to cooler ones, but again, the information analysis is fundamental (it's even needed to define what temperature is).

| cite | improve this answer | |
  • $\begingroup$ You've confirmed how it is taught by saying "they typically discuss it in terms of maximising the number (typically denoted W or Ω) of microstates per macrostate" but I don't see your argument for how this is fundamentally an information phenomenon rather than a phenomenon that results from the fact that it's easier to maximize the ways energy is distributed than to attain a specific, non-maximal energy distribution. $\endgroup$ – SuchDoge Apr 10 '18 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.