In the framework of information entropy, one common question is how to develop a code that minimizes the cost of transmission of a message, given that each bit as a fixed unit price. If the distribution of words is known, a Huffman code will then provide the required minimum. The price paid for the message is then proportional to the information entropy of the message.

I'm interested in understanding how this framework can be applied in the context of thermodynamic entropy. Can entropy (or rather, the product TS) be seen as the minimum amount of energy required to measure the micro-state of a system, given its ensemble? If this is the case, why does measuring a high-temperature state cost more energy?

  • $\begingroup$ If you want to completely measure the state of the system, then the information entropy of that measurement is $H = S/(k_B\ln 2)$ bits. If you want to convert that into an energy via Landauer's principle, you should multiply by the temperature of the heat bath to which your measurement/computation equipment is connected. But that need not be the same as the temperature of the system you're measuring. So it's $T_\text{measuring device}S_\text{system}$, not $T_\text{system}S_\text{system}$. $\endgroup$
    – N. Virgo
    Mar 3 '20 at 10:48
  • $\begingroup$ Thanks for pointing out Landauer's principle. Is there a way to understand it in a statistical way? Assuming for example we have a particle whose velocity is a random variable following Boltzman's distribution, can a first-principles estimate be derived for the measurement energy? $\endgroup$
    – Whelp
    Mar 3 '20 at 11:45
  • $\begingroup$ Landauer's principle is an inherently statistical thing. See my answer here, and perhaps also read up on "Szilard engines" and Maxwell's demon to understand the relation to stat mech. In regards to measurement, the issue is really that if you want to keep measuring repeatedly, you will eventually have to erase old data to make space for new measurements. It's erasing this information that has an energy cost. In principle, everything else can be done reversibly. $\endgroup$
    – N. Virgo
    Mar 3 '20 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.