Does entropy measure extractable work?

Question

Entropy has two definitions, which come from two different branches of science: thermodynamics and information theory. Yet, they both are thought to agree. Is it true?

Entropy, as seen from information theory, measures our ignorance about a system. Entropy, as seen from thermodynamics, measures the amount of extractable work. One is epistemological, the other is ontological. Yet, they both can agree, if entropy really measures the amount of extractable work, given one's knowledge about the system.

But, is this true? Does Shannon's entropy computed on the probability distribution of a physical system express the amount of work we can obtain from it? How do increases in our knowledge increase the amount of extractable work? Is Landauer principle powerful enough to establish the connection?

Answer 1

UPDATE: Below I am answering yes to the first question in the post (are the two kinds of entropy the same up to a constant). This has led to some confusion as both Matt and John gave answers saying "the answer is no", however I believe they are referring to the title "Does entropy measure extractable work?". Although the author uses the two interchangeably, the question itself contains a false premise: namely that physical entropy is a measure of extractable work ("Entropy, as seen from thermodynamics, measures the amount of extractable work"). This is simply not true in the general case, though can be true for certain special cases, as Matt's counter example shows. A concrete case of this is a ball placed anywhere on a level surface. If the ball is placed randomly, no more work can be extracted than if its location is known.

---

The answer is yes, the two kinds of entropy are the same up to a constant factor of $k_B \log 2$ (which is also the origin of Landauer's principle). On way to see this is via Maxwell's demon. In particular, via the Szilard engine, an idealised heat engine that uses a single particle gas. You then introduce a partition, which effectively partitions the gas into two regions, only one of which contains the particle. Now, if you knew which side of the partition the particle was on, you could use the pressure difference to extract work, and if you don't you can't since you don't know which way it will push.

Now the connection with information theory comes in when we measure which side of the partition the particle is on. From this we are gaining a certain amount of entropy (and hence information) in the register which holds our measurement result. But having this information decreases the entropy of the gas. And hence you can go back and forth between information entropy and physical entropy.

There is a fairly extensive literature on the matter, so instead of trying to give you a list, I'll point you to a review article on Maxwell's Demon and information theory from a few years ago: arXiv:0707.3400,

Answer 2

The answer is no. Consider a system that has a degenerate ground state, such that the density matrix is a mixture of two ground state eigenstates. This has nonzero Shannon entropy, but you can't extract any work from it. More generally, thermodynamic entropy is not really well-defined for nonequilibrium systems, but Shannon entropy is.

My own take is that thermodynamic entropy and Shannon entropy are two conceptually distinct things. They happen to coincide in a wide variety of circumstances, but not always. It is not even clear to me whether the cases in which they happen to coincide in classical and quantum theory are necessary coincidences. It may be possible to come up with a well-defined physical theory in which they never coincide, e.g. in the convex set famework for generalized probabilistic theories studied in the quantum foundations community.

Answer 3

Because the word "work" has multiple meanings, the answer in general is "no".

Typically the first meaning taught to students is thermodynamic, but (as Dirac demonstrated) a generalized meaning (which includes thermodynamic work as a particular case) amounts to:

Given a class of dynamical processes, 'work' is any potential function that naturally describes what that class accomplishes.

Specifically, in the context of isotope separation, Dirac established that the work potential $\mathcal{V}_c$ that is naturally associated to an isotope concentration $c$ is given by

$\displaystyle\qquad \mathcal{V}_c(c) = (2 c' - 1) \log\left[\frac{c'}{1-c'}\right]$

or equivalently for spin polarization $p = 2 c - 1$ the Dirac value function $\mathcal{V}_p$ associated to separative transport of spin polarization is

$\displaystyle\qquad\mathcal{V}_p(p') = \mathcal{V}_c(c')\big|_{c' = (1+p')/2} = p' \log\left[\frac{1+p'}{1-p'}\right]$

The key point is Dirac's value function is not proportional to an entropy difference (as is evident because $0\le \mathcal{V}_p(p') \lt \infty$ while per-mole entropy ranges over a finite range).

Moreover, the Dirac work associated to separation cannot be reversed to return mechanical work, since the separation process is entropically irreversible. Nonetheless, Dirac work has substantial economic value, and in fact defines the unit of value of a global market in separative work units (SWUs, pronounced "swooz").

For a derivation of the Dirac work function, see Dirac's own (unpublished) technical note "Theory of the separation of isotopes by statistical methods" (circa 1940), which appears in his Collected Works, or alternatively Donald Olander's survey article "Technical Basis of the Gas Centrifuge" (1972), or in general any textbook on isotope separation.