How is information defined from a thermodynamics point of view ? I came across some definitions using the concept of free energy of a system. If I have information stored in a finite volume of space what does it mean for the free energy and entropy of the space ?
$\begingroup$
$\endgroup$
1
asked Jul 8, 2011 at 20:43
-
$\begingroup$ Information doesn't really make sense in the thermodynamic point of view. It only has a meaning in the statistical physics point of view. If you have perhaps Shannon entropy in mind, that too is not a thermodynamic concept (rather, thermodynamic entropy is a special case of the Shannon entropy when subjected to equilibrium conditions). Also, what kind of system are you considering? The amount of entropy (or information) varies widely among different systems (e.g., there are systems with non-zero entropy at zero temperature, breaking the third law of thermodynamics). $\endgroup$– MarekCommented Jul 9, 2011 at 17:42
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Information is the negative of the entropy. The information you have stored in the system is the degree to which its entropy is not maximal.
For example, if you have N particles in a box, and you can tell each of them apart from one another, and there is a partition in the center of the box, you can store N bits of information by putting each of the particles on one side or the other of the partition. The entropy gain by lifting the barrier is N\log2, and this is N nats of information lost.
If you have a computer memory with N bits of data stored, the fact that you can read and amplify the data implies that the entropy of the computer is N nats below the entropy it would have in thermal equilibrium. If the computer has capacitors charged up to store a bit, then this capacitors must have at least log2 less entropy than it should at the ambient temperature.
These bounds are ridiculous underestimates for real computers. The actual storage devices are so far away from thermal equilibrium that they have many many nats per bit, even in the most efficient computer imaginable. But the principle is that the thermal equilibrium state is, by definition, the state where you have no information about the system beyond the conserved quantities or their conjugate variables. You know the energy/temperature, the volume/pressure, the number-density of species/chemical potential, but you know nothing else, so the system is in a Maxwell Boltzmann maximum-entropy/minimum-information distribution.
The relationship between the definitions of entropy and information was noted and understood by Shannon, and led to a reevaluation of the foundations of thermodynamics in the 1960s-1980s. The physical nature of information has been emphasized many times by Landauer.
