# Is physical entropy opposite to information entropy? [duplicate]

I am currently taking a class on Decision Theory where I was introduced to the notion of "entropy" in an information theoretic setting while we were studying decision trees. I was given the following example:

Imagine that you have three sacks - one with four red balls, one with three red and one blue ball, and one with two red and two blue balls. In an information-theoretic way, the first sack has minimum entropy because all the balls are alike hence maximum information; the second one has higher entropy than the first sack while the last one has maximum entropy because you are completely unsure about the information.

To me, this goes against the intuition of entropy from a thermodynamics point of view. The second law of thermodynamics asserts that the entropy is increasing and this will result in the Heat Death of the Universe when all the molecules will have the same temperature and we will not be able to extract any new energy from them. So comparing to the sack example, this seems like that all the molecules at this point of the Universe will have the same temperature so they have the lowest information entropy. However, according to the 2nd law of thermodynamics, the entropy will be at maximum during this stage.

I have always understood the thermodynamic entropy as a way to measure how "similar" different microstates are. The more similar, the higher the entropy. But the information theory definition is the exact opposite. I would love any insight on this apparent contradiction.

## marked as duplicate by Rococo, GiorgioP, John Rennie thermodynamics StackExchange.ready(function() { if (StackExchange.options.isMobile) return; $('.dupe-hammer-message-hover:not(.hover-bound)').each(function() { var$hover = $(this).addClass('hover-bound'),$msg = $hover.siblings('.dupe-hammer-message');$hover.hover( function() { $hover.showInfoMessage('', { messageElement:$msg.clone().show(), transient: false, position: { my: 'bottom left', at: 'top center', offsetTop: -7 }, dismissable: false, relativeToBody: true }); }, function() { StackExchange.helpers.removeMessages(); } ); }); }); Mar 7 at 7:42

• You may find useful the answers to this previous question: physics.stackexchange.com/questions/375904/… – GiorgioP Mar 6 at 22:50
• Atoms do not have a temperature. – my2cts Mar 6 at 22:52
• > "this will result in the Heat Death of the Universe when all the molecules will have the same temperature and we will not be able to extract any new energy from them." Where did you read this? It is a simplistic extrapolation of kinetic theory of gases to large scales and distant times that is very hard to justify. On the scale of galaxies, gravity makes equal temperature very unlikely, such a state would have low entropy. – Ján Lalinský Mar 7 at 1:17
• I think that you should look at the statistical definition of thermodynamic entropy to resolve your confusion. en.wikipedia.org/wiki/Entropy#Statistical_mechanics . – anna v Mar 7 at 4:47

Molecules in a volume of gas where we know only temperature, volume and number of particles, have unknown positions and momenta, or other microscopic states. This lack of knowledge is what makes information entropy for given $$T,V,N$$ large. In physics, the maximum possible value of information entropy consistent with $$T,V,N$$ gives thermodynamic entropy of the system.