A simple argument to go from entropy to energy?

Question

In many analytically solvable statistical physics models it is possible to define, derive and calculate entropy and thus accurately observe how entropy relates to very important things such as phase transitions, phase separation, condensation and so on.

One insight conveyed by these models is that the interplay between the affinity of thermodynamic systems to statistically favored high entropy states and their tendancy to lower energy is very important.

One thing I still have a hard time to do after going through so many derivations is to find a good argument in good simple words how a number of configurations (i.e entropy) can be converted to a number of units of energy (i.e heat) ?

Let me formulate the question in another way. We take a simple system such as a binary alloy or an Ising lattice. We expect the system to have two extremes : at high temperature the system is disordered and energy does not matter and at low temperature the system is ordered and entropy does not matter. Thus to estimate the transition temperature we count how many configurations the system has (as function of temperature) and we count the internal energy (as function of temperature). Now intuitively we should see the transition when these two counts are equally important, or have the same value, but the problem is that they do not have the same units. So how can we compare ?

Boltzmann's constant is used to define entropy, entropy is multiplied with temperature, and this gives us the units of energy. But to me this somehow feels like an empirical result.

Actually, many people say that Boltzmann constant is not natural constant, or not fundamental. If that is the case, how on earth do you convert your units of configuration counts to units of energy ?

Here is another way to ask : In the basic equation of free energy F = E - TS, how do you justify the multiplication of the "energy" T with the number of configuration ? In a mechanical system without thermal fluctuations (such as a cable fixed from its two ends and hanging in gravity) the macroscopic observable equilibrium is determined by the minimum energy configuration. For a thermodynamic system it is expected that the number of configurations gets added to the internal energy to find thermodynamic equilibrium, but how do we jutify the structure of Internal Energy - Kinetic Energy × Configuration Number ? Why is this the "optimal" structure ?

Answer 1

You're assuming that entropy must be totally unrelated to energy. This is not true: conceptually, it very much is - at least, one sense of entropy very much is related to energy.

The equations is not

$$S = \Omega$$

it's

$$S = k_B \ln \Omega$$

so the entropy is a physical quantity that is proportional to the logarithm of the number of microscopic configurations that can realize a given macroscopic state. But proportionality does not mean identity, much less conceptual identity!

Another example of this is Einstein's famous relation, $E = mc^2$. Some take this as saying that "mass is energy" (and you can take units $c = 1$ and then claim that $E = m$, which is no different from taking $k_B = 1$), that there is no distinction between mass and energy. But I would say that's quite wrong. Conceptually, they are very different things. Energy is a sort of "natural 'money'" that you have to hand around to effect physical change; while mass is a measure that characterizes the responsivity of an object to an applied force. To just elide the distinction is actually to trivialize Einstein's result, because it is precisely this seeming absolute alterity between them that makes the result profound in the first place! It's saying that the "natural 'money'" is intimately bound up with inertia, which is not a surprise one might have expected first off, but turns out to be true specifically about how our world is constructed!

Likewise, for entropy. Entropy is a physical quantity that can either be understood as measuring energy "wastage", in the sense that it tracks how much energy has been "degraded" or rendered "unavailable for use" to a heat engine; or else you could say energy "dispersal", but one has to be careful about the type of dispersal meant since it isn't necessarily a purely spatial dispersal in all cases - instead it's more a dispersal amongst different aspects or energy "bins" within the system, which leads to more possible combinations, hence a higher $\Omega$. In any case, entropy very much involves energy! It's essentially mandated by the second law, since the second law talks of energy, and talks of how "something" has to change about a system to render energy more and more "lost" as it is cycled around repeatedly. Quantifying that "something" is the core job of entropy from a thermodynamics perspective.

Presumably, those arguing for a dimensionless entropy are perhaps suggesting that we should say entropy has nothing to do with the energy at all, but are jiving at it from some sort of informational perspective. But that's, again, much like Einstein's equivalence: it's a result that's actually deep, not a mere definition or tautology! Informational entropy is a radically different concept from thermodynamic entropy: it's the amount by which an agent receiving a message gains "confidence" in what the state of affairs of some external situation is. There is no mention of energy at all!

And what should be surprising is that these two conceptually very different things tend to be proportional in the same way as $E = mc^2$! In fact, the $S = k_B \ln \Omega$ kinda gives it away too readily; if you want, the real shocker statement is

$$\frac{1}{k_B} \oint_\gamma \frac{\delta Q_\mathrm{rev}}{T} = -\sum_i P_i \ln P_i$$

which puts the actually thermodynamic definition of entropy on the left, and the fully informational definition of entropy on the right, with suitably interpreted probabilities $P_i$. We have an integral involving heat and energy on the left - steam, moving engine parts - and then on the right, some very abstract, computer-like process. Entropy, in this form, is an answer to the question "what does the boiler rig of a steam locomotive have in common with your iPhone?" which, on the face of it, seems utterly laughable!

So what I'm saying is, there's very good reason to give it energy units, and there's very good reason to distinguish things conceptually that should be distinguished, and not simply try to lump everything together into neat and oversimplified packages, and then note they bear a deep connection and correspondence. Especially given that if you are not careful, you may draw incorrect conclusions from the above relation, such as that if you shuffle up a bunch of cards, that increases its thermodynamic entropy, which is simply not true - at least, not because of changing the order of the cards! (It will almost surely increase it by heating it up, though, due to friction and plastic dissipation!)

Answer 2

To see the connection between information (or entropy) and energy, consider the following system. We'll show that knowing information about the system implies that you can do useful work, but erasing information in the process, and that erasing the information increases entropy. Then, we'll talk about how this might be related to "free energy."

Start with an empty box of volume $V$ and place a membrane dividing it straight down the middle into two equal parts $L$ (left) and $R$ (right). Then place a single classical particle in the box on the left side and allow it to equilibrate to temperature $T$ by coupling it to a reservoir.

Because we know for sure the particle is in state $L$, the entropy $S_\text{LR}$ associated with the uncertainty in the state $L$ or $R$ is zero,

$$ S_\text{LR} = 0. $$

The key here is that knowing that the particle is in state $L$ allows you to do useful work with it.

Here's how. Put a piston up against the membrane (coming in from the right, and doing no work in the process), then remove the membrane, and finally allow the one-particle "gas" to expand isothermally from volume $V/2$ to the full volume $V$.

The amount of work the gas does on the piston is

$$ \Delta W = \int p dV = kT \int_{V/2}^{V} \frac{dV'}{V'} = kT\ln(2). $$

Because $T$ is constant, the internal energy $U$ of the particle stays the same throughout this process, $\Delta U = 0$ (because we're dealing with an ideal gas.) Heat from the reservoir has been entirely converted into work. Converting heat directly to work with no other side-effects isn't allowed, so the entropy must have increased elsewhere.

In fact, it increased in the gas. Since the particle can now either be in $L$ or $R$ (there's no dividing membrane anymore, so it can freely move between the two) its entropy has increased by

$$ \Delta S_\text{LR} = k \ln (2). $$

The punchline is that the relatively low entropy of the original system gave us the ability to convert heat into work, but the system absorbed the extra entropy, erasing some information we had about it.

This example above is general. If we fix the internal energy and temperature, we can always let a system do some work isothermally, absorbing some entropy from a thermal reservoir in the process, and we will do work $ T \Delta S$, having erased some information in the process.

In general, as $S$ is the erased information in a system, $TS$ is the energetic equivalent of that used up, erased energy. This energy can't be converted to work.

This might motivate you to define a quantity which is "the usable work in a system at some temperature," which gives you

$$ F = U - TS $$

also called the free energy.

Answer 3

I upvoted the answer already here by The_Sympathizer because I think it makes some very good and relevant points. I am not offering a complete answer, just some further thoughts.

This is quite a subtle area, more subtle than many people realise. The subtlety is in counting the number of microstates $\Omega$ which is what you have to do if you want to get at entropy when modelling a system by statistical mechanics. We count mutually orthogonal energy eigenstates, but why? Why should that be the number which cannot decrease with time for an isolated system? Why isn't it just constant with time? etc. The reason is to with internal dynamics, and the way the motion among the microstates tends to visit mutually orthogonal states with equal rates. This is often glossed over rather quickly in introductory courses on statistical mechanics.

The other equation you mention is $F = U - T S$. This comes from thermodynamic arguments, where it is possible to show that the state-function defined this way is the one which is maximised, in equilibrium, for a system exchanging energy with a thermal reservoir. In this equation we employ the thermodynamic definition of temperature, which leads to the relation $$ \frac{1}{T} = \frac{\partial S}{\partial U}. $$ There is no need to regard temperature as a form of energy. It is the inverse of a rate of change of entropy with energy.

Having said that, it can be useful to define temperature units (in this area of physics) in such a way that Boltzmann's constant $k_B$ comes out having a value 1. That does not necessarily mean it is dimensionless. It means it has a value of one energy-unit per temperature-unit. It is only by insisting that this constant is dimensionless (with the result that so is entropy) that one arrives at temperature and energy being the same sort of physical quantity (in terms of physical dimensions that is), but they remain different kinds of physical quantity really, no matter what one says about units and dimensions.