# How can the maximum entropy and minimum energy principles be (physically) equivalent?

In Chapter 5 of his famous textbook on thermodynamics, Callen argues for the "equivalence" of the maximum entropy (Max-Ent) principle and the minimum energy (Min-En) principles. I quote from Callen first:

Entropy Maximum Principle. The equilibrium value of any unconstrained internal parameter is such as to maximize the entropy for the given value of the total internal energy.

Energy Minimum Principle. The equilibrium value of any unconstrained internal parameter is such as to minimize the energy for the given value of the total entropy.

As far as I know (though Callen never makes this explicit in what, I think, represents an error or at least an oversight) the Max-Ent principle applies if and only if the (composite) system under investigation is isolated. On physical grounds, an isolated system is always characterized by fixed energy $$U$$. It's with that background that I ask the following.

Callen first gives a "qualitative" proof (in one direction of the equivalence) which goes as follows:

Assume, then, that the system is in equilibrium but that the energy does not have its smallest possible value consistent with the given entropy. We could then withdraw energy from the system (in the form of work) maintaining the entropy constant, and we could thereafter return this energy to the system in the form of heat. The entropy of the system would increase ($$dQ = TdS$$), and the system would be restored to its original energy but with an increased entropy. This is inconsistent with the principle that the initial equilibrium state is the state of maximum entropy! Hence we are forced to conclude that the original equilibrium state must have had minimum energy consistent with the prescribed entropy.

Aside from the fact that one seems to be applying the Max-Ent principle to a non-isolated system in this "argument", I have an even bigger objection to the first line, wherein we are supposing that we are at some fixed/constrained entropy!

The "mathematical argument" seems to suffer (at least it seems to me) from that same shortcoming. See here for the very nice answer from Chemomechanics which reproduces Callen's derivation. Translating the result of that proof into words, we seem to have shown that (assuming a simple $$U,V,N$$ system for simplicity) if on the surface $$\psi(S,U,V,N) = 0$$ we are at a point such that $$S$$ is locally maximum at the given $$U$$, then that point is also locally a $$U$$ minimum for that given $$S$$ (i.e. the $$S$$ corresponding to the max entropy).

My problem with the above is that Callen goes on (or seems to go on) to claim that the two principles are in every way equivalent, but this seems absurd given that each principle (on my reading of things) is applicable in completely different contexts. Callen for example analyzes the diathermal piston problem using both principles, which makes no sense to me given that the composite system of two gases is either at fixed energy or at fixed entropy (though I'm not sure how one would arrange the latter) -- but not both!

To adapt a clever (if controversial) phrase, this question can be summed up as "don't the Max-Ent and Min-En principles have nonoverlapping magisteria and, if so, how does it makes sense to talk about their being equivalent?" I am hoping someone can elucidate exactly in what sense these are equivalent. There have been many questions asked on this site to that effect but none which suitably address this last point, I think.

Edit: Please consider the following. Perhaps if someone can point out the error in this deduction it will point out where my misunderstanding arises from.

Suppose I give you an isolated system at some $$U_i$$ and tell you that some internal constraint has been released. By Max Ent, you can then search in configuration space (as restricted to the $$U_i$$ constraint) for the state which maximizes $$S$$ on this plane. Suppose I find that this final state has $$S_f>S_i$$ But if I use Min En as the method as applied to the same system, how do I proceed? The initial state has some entropy $$S_i$$ -- do I search on this plane of $$S = S_i$$ for the minimum energy state (that seems to be what Min En says)? And, if so, since this minimum energy state will have $$S = S_i < S_f$$ (and so cannot be the same state as I just found with the maximum entropy principle), surely (though of course I have erred somewhere since all of you, Callen, and Gibbs disagree with me!) I have just shown that the minimum energy principle has led me to a different conclusion than the maximum entropy principle?

• what makes you think he is applying it to a non isolated system? Commented Jun 26, 2023 at 15:07
• @BobD I asked about it elsewhere and that was the consensus. But just on physical grounds, Max Ent always applies only to isolated systems (at least a priori)?
– EE18
Commented Jun 26, 2023 at 15:23
• But I see nothing that implies a non isolated system. Commented Jun 26, 2023 at 16:16
• @BobD Two things: 1) the min energy principle (seems to) requires constant entropy which doesn’t make sense for an isolated system and 2) in Callen’s proof he explicitly uses some other systems to do the work on and to deliver the heat.
– EE18
Commented Jun 26, 2023 at 17:28
• 1) The entropy of an isolated system where all the properties are in equilibrium is maximized. In the absence of any disequilibrium no entropy can be generated. Since the system is isolated, no entropy can be transferred (by heat). 2) I think he is exchanging work and heat with the surroundings to show, by contradiction, that the initial equilibrium state must have had minimum energy since, given no change in internal energy (Q=W) there would be an increase in entropy. Commented Jun 26, 2023 at 21:09

(1). A system (at equilibrium) will maximize its entropy when it is isolated - that is, with $$U$$ and $$V$$ held constant.

(2). A system (at equilibrium) will minimize its internal energy when $$S$$ and $$V$$ are held constant.

To see this, I'll consider a general setup of a system $$S$$ and an environment $$E$$, and I'll only assume that the parameters of the environment $$(P_E, T_E, ...)$$ are fixed a priori. Suppose heat $$dQ_S$$ is exchanged to the system - then the first law of thermodynamics implies $$dQ_S=dU_S - (dW_S-P_E dV),$$ where $$dW_S$$ is the work we extract from the system, and $$P_E dV$$ is the work the environment does on the system. By the second law of thermodynamics, $$dS_S - \frac{dQ_S}{T_E}\geq 0 \implies dQ_S \leq T_E dS_S.$$ Then, substituting gives $$dW_S \geq dU_S+P_E dV-T_EdS_S.$$ Some systems can use the expansion of gas to produce useful work (like a gun, which uses the expansion of gas to accelerate bullets). However, I'll assume we leave the system alone and don't extract any work from it (we "mechanically isolate" it), so that $$dW_S=0$$. Then the last equation reads $$d(U_S+P_EV-T_ES_S)\leq0,$$ which is about as general a minimization principle you can get without introducing chemical potentials. One can then read off (1) and (2) as special cases of the above, as well as the following results:

(3). A system will minimize its Helmholtz free energy $$F\equiv U-TS$$ at equilibrium when temperature and volume are held constant.

(4). A system will minimize its Gibbs free energy $$G\equiv U+PV-TS$$ at equilibrium when pressure and temperature are held constant.

• I have a number of questions, but will stick with just two. The first has nothing to do with the crux of my question but still leaves me confused: why is the $P_E dV$ term there? I thought you were applying the first law to energy flows with respect to S, but it seems not? More important though is (2): you seem to have shown that the principle of minimum energy follows (in a certain context) from the Max Ent principle, but not that it implies (Max Ent). Is this fair to say and, if so, why does Callen insist on saying they're equivalent?
– EE18
Commented Jun 26, 2023 at 20:57
• 1. I split up the work into two components - the work done by the system on the environment, $dW_S,$ and the work done by the environment on the system, $P_E dV.$ That's also why they have different signs. 2. Callen seems to be playing it loose with the language a little - the correct interpretation is that both max E and min U are special cases of maximizing the entropy of the universe. They aren't equivalent in the way you framed them because they cover different conditions. Commented Jun 26, 2023 at 21:53
• 1. But why would that be a factor in the energy balance? Doesn't the work done by the environment necessarily equal the work done on the system? 2. Thank you for clarifying; I am going to leave it for a bit to think on this, but I think I agree that ultimately Callen is exaggerating the extent to which Max Ent and Min En are equivalent. I think Min En should be thought of in the sense you do -- as the appropriate minimization criterion in a particular instance.
– EE18
Commented Jun 27, 2023 at 1:13
• This is not at all a solution to what the OP is trying to ask. You are saying that both Max Ent and Min En are consequences of a more general unifying principle, namely that grand thermodynamic potential is minimised or grand massieu functions are maximised. That is a new thing to postulate, and nothing to do with the postulates that Callen is working with, that OP wants to know the equivalence between two types of statements. Commented Jun 27, 2023 at 2:59
• @naturallyInconsistent I think Roger Yang is correct: the only principle in Roger's derivation is the second law, that's the inequality in Roger Yang's second equation. All other inequalities follow, so I don't see two governing principles (inequalities), only one. Commented Jun 27, 2023 at 15:55

As @Roger Yang has pointed out, maximization of $$S$$ under fixed $$U$$ and $$V$$ is equivalent to minimization of $$U$$ under fixed $$S$$ and $$V$$. This follows from the math but here I want to offer not a proof but a demonstration through an example.

The standard example to demonstrate entropy increase under fixed $$U$$ and $$V$$ is free expansion against vacuum: a thermally insulated box is half filled with a gas and half evacuated, the wall that separates the two parts is removed and the gas expands to fill both sides of the box. No energy has been exchanged with the surroundings, therefore $$U=\text{const}$$. And since the gas has larger volume to roam, entropy increases.

Now, let's perform this expansion under fixed entropy. To do so we must extract work from the system, so we place a spring that opposes the expansion and conduct the process reversibly, since reversible plus adiabatic guarantees isentropic. Since the system produces work its $$U$$ decreases. And since the process is reversible, it produces the maximum possible amount of work, which means that the system ends up with the minimum possible internal energy.

Added in response to the edit by the OP

The question, as I understand it, is to come up with a process that demonstrates the minimization of $$U$$ when $$S$$, $$V$$ and $$N$$ are constant.

Here is how we will do this:

The system is divided into two parts, part 1 with $$(S_1,V_1,N_1)$$ and part 2 with $$(S_2, V_2,N_2)$$ such that $$S_1+S_2$$, $$V_1+V_2$$ and $$N_1+N_2$$ are fixed. We will exchange entropy between the two parts keeping all other variables constant. Let's say that an amount $$|d S|$$ is transferred from 1 to 2. We do this as follows: From system 1 we remove an amount of heat $$|dQ_1| = T_1 |dS|$$ To system 2 we add the amount of heat $$|dQ_2| = T_2 |dS|$$ The net effect is that entropy has moved from part 1 to part 2 with the total entropy staying the same.

The entropy generation in the universe is the sum of the entropy change in 1, plus the entropy change in 2, plus the entropy change of the bath: $$dS_\text{gen} = -|dS| + |dS| + \frac{+|dQ_1|-|dQ_2|}{T_\text{surr}} \geq 0,$$ from which we conclude that $$+|dQ_1|-|dQ_2| \geq 0 .$$ By first law applied to the system, noting that there is no work since volume is constant, we have $$dU = -|dQ_1|+|dQ_2| \leq 0$$ Conclusion The maximization of entropy in the universe (system plus surroundings) is equivalent to minimization of $$U$$ in the system under the condition that $$S$$, $$V$$ and $$N$$ of the system remain constant.

The last result we can write as $$dU = -T_1|dS| + T_2|dS| \leq 0.$$ At equilibrium $$dU=0$$ because $$dS_\text{gen}=0$$, and since this ought to hold for any small $$|dS|$$, we must have $$T_1=T_2$$ Therefore, the minimization of $$U$$ leads to the the same criterion for thermal equilibrium as the maximization of $$S$$.

You can guess that that the same analysis applied to $$V$$ and to $$N$$ will yield the equilibrium conditions $$P_1=P_2$$ and $$\mu_1=\mu_2$$.

The bigger lesson The minimization of potentials should always be understood to be with respect to the partitioning of the extensive variables of the potential between parts of the system while keeping the intensive variables constant. Here, $$U=U(S,V,N)$$ has three extensive variables. We demonstrated the minimization of $$U$$ with respect to partitioning one of them, $$S$$. You should be able to show the same under partitioning of $$V$$ or $$N$$.

• The part I don't understand is what you (and Callen) mean precisely when you say "is equivalent to". How can two things be equivalent when they apply in different instances?
– EE18
Commented Jun 27, 2023 at 1:15
• In my answer equivalence is established as follows: "Maximization of entropy at fixed $U$ and $V$" leads to the statement "the maximum work is obtained under reversible process" and this in turn leads to the statement "$U$ is minimized when the system undergoes reversible process under constant $S$ and $V$". Each of the three statements implies each other, therefore, they are equivalent. In fact, all of them are alternative statements of the second law. See Roger Yang's answer for the mathematical proof. Commented Jun 27, 2023 at 2:06
• If you really read Roger Yang's argument, he is not at all answering the OP, because he is asserting that both relationships are consequences of a deeper principle, one that is not postulated by Callen. Your argument actually does better than Roger's. Commented Jun 27, 2023 at 3:01
• @EE18 Indeed, I agree with Themis. Min $U$ and max $S$ are equivalent in a mathematical/logical sense, in that one can be derived from the other (though, it requires the second law of thermodynamics, which is a bit roundabout considering that both statements are really equivalent to the second law too). However, I meant that they aren't equivalent in a practical sense, since the problems which can be solved with min $U$ can't be solved with max $S$, and vice versa (in contrast to, say, the equivalence between Newtonian and Lagrangian mechanics). Commented Jun 27, 2023 at 19:30
• As I said to Roger, I will have to reread Callen at some point this weekend as I still think I don't understand. Thank you as always for your help, Themis!
– EE18
Commented Jun 28, 2023 at 1:28

This is a misunderstanding on your part. Callen and Chemomechanics is precisely correct.

It should be obvious once I introduce the notation here, but I will be explicit. Let smaller letter subscripts s and r refer to the scrutinised system and the combined heat and work reservoir, and that the isolated thing be the combination of s and r.

Then $$U_s+U_r=\text{const}$$ as it attains the maximum combined entropy $$S_s+S_r$$. If $$U_s$$ is not the minimum value when the combined entropy is maximised, then you can keep $$S_s$$ (and $$S_r$$) constant as you extract some work, and then rearrange some $$S_r$$ into $$S_s$$. This conflicts with the version of the statement of the 2nd Law in which you cannot extract purely extract work from an isolated system without a change in entropy. Or you can find a 3rd system to take heat or entropy from, and then the combined entropy can rise yet again, giving rise to the original contradiction that was written in Callen. Or you can stop at the rearrangement of some $$S_r$$ into $$S_s$$ step and note that you have just found a lower total energy state that also has the same maximal entropy. All three statements are unacceptable.

Therefore, Max-Ent implies Min-En. QED.

I have an even bigger objection to the first line, wherein we are supposing that we are at some fixed/constrained entropy!

You just agreed with Max-Ent. If it is already maxed, it should be fixed!

On my first reading I thought what is happening is that, on the manifold hypersurface of possible system states, the equilibrium is not pointing vertically (Max-Ent) or horizontally (Min-En), but rather diagonally. This is helpful later on as you work with other thermodynamic potentials and/or Massieu functions. But maybe I was also interpreting Callen wrongly. Apparently thermodynamics itself can prove this equivalence.

• As I said to Roger, I will have to reread Callen at some point this weekend as I still think I don't understand. Thank you for the discussion as always naturallyInconsistent -- this discussion from Callen (who is otherwise generally quite clear) has evaded me.
– EE18
Commented Jun 28, 2023 at 1:29
• Dear naturallyInconsistent: I have reread your and everyone else's answers, reread Callen, and still cannot understand. I have placed an edit in my OP. If you would kindly take a look and consider where the flaw in my logic comes from I would greatly appreciate it. Thank you!
– EE18
Commented Jul 1, 2023 at 21:44
• Your edit makes a wrong assumption. As you correctly stated initially, Max Ent and Min En have different operating conditions. In your isolated system, you cannot apply Min En because its U is fixed. That is a vertical slice in the state space, and Max Ent tells us that the equilibrium state will be found at the top. What Callen is telling you is that at this equilibrium state, if you take a horizontal slice, i.e. let U be unfixed at this state (and instead fix S), then it will be a Min En too. Commented Jul 2, 2023 at 5:55
• Note the bottom of the first page from Gibbs here: (physicsforums.com/threads/…). He clearly states that both principles apply to isolated systems.
– EE18
Commented Jul 2, 2023 at 14:49
• I will reread with this understanding, thank you again and as always! I'm very grateful to have people help me as I self-study since there's no one else to ask!
– EE18
Commented Jul 2, 2023 at 17:36