This question already has an answer here:
I am currently taking a class on Decision Theory where I was introduced to the notion of "entropy" in an information theoretic setting while we were studying decision trees. I was given the following example:
Imagine that you have three sacks - one with four red balls, one with three red and one blue ball, and one with two red and two blue balls. In an information-theoretic way, the first sack has minimum entropy because all the balls are alike hence maximum information; the second one has higher entropy than the first sack while the last one has maximum entropy because you are completely unsure about the information.
To me, this goes against the intuition of entropy from a thermodynamics point of view. The second law of thermodynamics asserts that the entropy is increasing and this will result in the Heat Death of the Universe when all the molecules will have the same temperature and we will not be able to extract any new energy from them. So comparing to the sack example, this seems like that all the molecules at this point of the Universe will have the same temperature so they have the lowest information entropy. However, according to the 2nd law of thermodynamics, the entropy will be at maximum during this stage.
I have always understood the thermodynamic entropy as a way to measure how "similar" different microstates are. The more similar, the higher the entropy. But the information theory definition is the exact opposite. I would love any insight on this apparent contradiction.