Is concept of entropy really indispensable? Especially when the concept of potential energy can serve the purpose?

Question

We see that all the natural systems aspire for minimum potential energy state and we also see that all the natural systems also aspire for maximum entropy state. Now from this understanding it seems that entropy is inversely proportional to potential energy. All the systems aspire for minimum potential energy and maximum entropy. It seems like the concept of potential energy itself and in isolation sufficient to explain all the phenomena which we explain with the help of concept of Entropy, so why do we need an extra concept of Entropy when the concept of potential energy can serve all the purposes?

Can we completely define concept of Entropy in terms of concept of potential energy?

What extra knowledge the concept of Entropy serves us in addition to the concept of potential energy?

Answer 1

We see that all the natural systems aspire for minimum potential energy state

This is not true. For example, the earth's minimum potential energy state would be a state in which it was inside the sun.

What extra knowledge the concept of Entropy serves us in addition to the concept of potential energy?

The concrete sidewalk in front of my house was wet when it was first poured. Then the concrete dried and hardened. When it rains, the concrete doesn't reverse the chemical reaction and become soft again, nor will it do so even if I apply some heat, although it would be consistent with conservation of energy if it did so. We need the second law of thermodynamics to explain why this doesn't happen.

Answer 2

I love that you are asking about how these two similar principles relate. However, you assume that we can derive entropy maximizing from energy minimizing, and I don't think you can: I would instead derive the minimum energy principle from maximizing entropy. In my view, the minimum energy principle is fundamentally an entropy result and until you understand entropy, you won’t understand the minimum energy principle. So my explanation flows in the opposite direction of how you understand it.

Why not just energy?

To put it in harsh terms with an example, the minimum energy principle suggests that all the air molecules in the atmosphere should fall to the floor and stay there.^{[Note 1]} That is the minimum of gravitational potential energy, and they have certainly had years to fall. Why aren't they there yet? Clearly we should look for a more general principle than the minimum energy principle, of which the minimum energy principle is a special case. The air staying in the atmosphere would satisfy the broader principle, but not the minimum energy principle.

In softer terms with a constraint, the minimum energy principle must not make ultimate sense of the world due to the constraint that energy is conserved. Energy can't be created or destroyed, but only spread around various “degrees of freedom” where it can live. This already suggests something limiting about the minimum energy principle, because if you look at a big enough system the total energy must be constant: and there is no minimizing a constant. What is it about the energy of the basketball bouncing across the court that wants the energy to leave it? And why doesn't energy come back into it? One can point to the fact that friction does negative work, but one runs into the problem that this energy has to go somewhere.

How entropy solves this problem

The principle of maximum entropy says that we become more uncertain about the state of the whole world over time. If we don't know some number exactly, and that number affects other numbers in the dynamical evolution of the system, then we soon don't know those other numbers exactly: and so our uncertainties tend to multiply across the system. Two particles with uncertain momenta collide, their momenta afterwards are even less certain. Our idea about where the energy in a system is, becomes less and less accurate. Eventually, the energy is just randomly permuted across all of the degrees of freedom, and in the average it is roughly spread evenly across all the places it can be (in specific instances we may see a little more of it here, less there, but not terribly by much).

We call that state thermal equilibrium. We can measure something called the temperature by that average energy in each degree of freedom. All of that energy in all those degrees of freedom is then called thermal energy. We can typically only get at it by connecting two systems with different temperatures, and catching some of the energy as it flows from the hotter one to the colder one.

The minimum energy principle is, then, a specific case. It is the specific case of a “system” of a few well-defined degrees of freedom that hold a lot more energy than the temperature of an “environment” of a lot of other degrees of freedom, with which the system can share energy. In such a case, as we become more uncertain about the location of the energy in the world, more and more of it should be in the environment and less in the system, until the system is in thermal equilibrium with its environment and only has the thermal energy of that degree of freedom. That temperature will increase as this energy is absorbed, but if there are lots of degrees of freedom in the environment, it doesn't have to increase by much. That just depends on the situation. For a large object like a soccer ball rolling down a hill, those thermal jitters can easily pass beneath observation.

So if we have $E$ lumps of energy^{[Note 2]} that we can spread across $N$ degrees of freedom, we can describe the random distribution as a bit-string with $E$ zeroes representing units of energy and $N-1$ ones representing partitions between degrees of freedom, so 001000100110 means 2 lumps in the first box, 3 in the second, 2 in the third, 0 in the fourth, 1 in the fifth. The number of ways to arrange the system is then given by combinatorics as $$W=\frac{(E+N-1)!}{E!(N-1)!}.$$ So now imagine your situation: you have a couple of degrees of freedom that you can see directly: say, the kinetic and potential energy of a bouncing basketball. They have some energy which I will just call $U$. And then you have the degrees of freedom that you cannot see directly, all of the vibration modes of the floor and the sound in the air, which I describe with the above system and its energy $E$. And we have energy conservation: $U + E$ is constant. So as energy leaves the system $U$ and hence leaves our certainty, and flows into this environment, it increases this $E$.

Now if we compare $W(E)$ and $W(E-1)$ we will find that the most recent lump of energy flowing from $U$ into $E$, had the effect of multiplying $W$ by $W(E) = W(E-1)\cdot(E+N-1)/E.$ We can approximate this for usual systems^{[Note 3]} as $\approx 1 + N/E$ and we measure entropy with the logarithm of $W$, so it added to $s=\ln W$ an amount $\ln(1+N/E)\approx N/E$, assuming that the typical degree of freedom has many lumps of energy on average.

We said $T=E/N$ is our measure of the average energy in any degree of freedom and now we see that as energy $\delta E$ leaves our bouncing ball, it causes an entropy increase of about $\delta s=\delta E/T$. So the minimum energy principle is a special case of the maximum entropy principle. The limit holds when the energy in the system is much larger than the temperature of a big reservoir that it can share energy with, $U \gg E/N$ but this does not change the temperature too much as the system releases its energy, $U \ll E.$

Notes

Note 1: the rough height of the atmosphere can be estimated by assuming it all has the same density $\rho$ as it does at the ground, in which case a column of air of that density exerts a pressure $P = \rho~g~h$, so that $h = P/(\rho~g)$, which using the density of nitrogen gas at Earth's surface gives h ≈ 8 km, which is roughly correct; the troposphere ends maybe 9-17 km in the sky, and the vast majority of the air is in the troposphere—so we are only off by a factor of 2. If you were instead to do the calculation assuming the nitrogen gas became liquid nitrogen, you would find that the figure is only h ≈ 13 m, much thinner, before low-temperature attractions start to take over. Intuitively, this effect called temperature is the big difference between liquid and gaseous nitrogen. But there is another way to calculate this number of ~8 km that really seals the deal: remember that temperature is supposed to be the average energy in every degree of freedom? Well, that energy $k_\text B T$ for room temperature is typically about $k_\text B T\approx25~\text{meV}$, milli-electron-volts, where an electron-volt is the energy an electron gains if it moves through a potential difference in vacuum of 1 volt. The mass of a nitrogen atom is about $m=28~\text{amu}$, and therefore the average energy in the height degree of freedom should be $k_\text B T = m~g~h$, giving a sort of average height... of h ≈ 9 km.

Note 2: That Boltzmann’s theory of entropy requires us to consider things like energy to come in lumps, is something that caused a great deal of difficulty in the early development of the theory. Then a theory called quantum mechanics came around and it said that for real systems, energy really does come in lumps, saving the theory. Before that happened, in 1905, a young and then-unknown Albert Einstein pointed out that it really does matter how big the lumps are, by noticing that if there are lots of little lumps then their fluctuations tend to average out, but if there are a few big lumps then their fluctuations are more noticeable. During this he also provided a first inkling of a truly powerful result in this statistical physics called the fluctuation-dissipation theorem, which basically says that we can connect the rate at which this bouncing-ball is losing energy to its environment (its dissipation) with the amount of thermal noise it starts to have from the energy coming randomly back from the environment the other way (its fluctuation).

Note 3: just to give a rough scale of what we are talking about here, you might imagine that for a real system we have $N\approx 10^{23}$ given by the approximate scale of Avogadro’s number, while $E/N$ might be in the range of $10^6$ assuming a phonon frequency in the MHz, and the energy in your half-kg basketball might be $U\approx 10^{21}$ of these units. So in case it is not clear, the impact of absorbing all of that basketball kinetic energy on the temperature of the floor is approximately negligible and the thermal vibrations of the basketball are on the order of, whatever, $10^{-15}\text{ m}$ or a millionth of a nanometer fluctuation in its height: you would never notice such a thing.

Answer 3

For just one example:

Consider a box full of $N$ particles of an ideal gas in the absence of gravity. There is no useful potential energy here, in a dynamical sense - the particles of the ideal gas are non-interacting except possibly for hard-sphere collisions (no interaction means no interaction potential, and a hard-sphere potential is trivial and contains no minimum), and there's no external potential imposed on the system. "Minimizing the potential energy" isn't really possible because there's no applicable potential energy to minimize. Despite this, the entropy of the ideal gas is not only well-defined, but quite useful for the prediction of the system's behavior.

Answer 4

To follow on with @probably_someone's answer replace the "potential energy" in the question by "internal energy", and note that the equilibrium entropy maximum principle is equivalent to the internal energy minimum principle. Does this mean that we can dispense with the entropy concept? And the answer to that is no, because while one can be derived from the other they refer to different physical situations: in the internal energy minimum case the entropy is fixed while in the maximum entropy case the internal energy is fixed.

Answer 5

Note that you are making an error of category. The entropy is always the property of an ensemble of systems, not of a single system, while the potential energy is the property of a single system (and an ensemble of systems only has an average potential energy). We can work with the entropy of a system in macroscopic thermodynamics, because the systems we consider there are big, so we can in a sense average over small parts of the system or because the molecular dynamics are fast, and we are interested in comparatively slow time-scales (and can therefore average over time).

For another example to what extent potential energy and entropy are not interchangeable, consider a system consisting of a single spin 1/2 in a magnetic field along $\vec e_z$, an ensemble of such systems has the entropy (where $p_{\uparrow/\downarrow}$ is the probability that the spin points up respective down for one system chosen randomly from the ensemble, $p_\uparrow + p_\downarrow = 1$, note that I work in a system of units where $k_B = 1$): $$ S = - p_\uparrow \log(p_\uparrow) - p_\downarrow \log(p_\downarrow) = -p_\uparrow \log(p_\uparrow) - (1 - p_\uparrow) \log(1 - p_\uparrow) $$ The average potential energy however is $$ U = \alpha (p_\uparrow - p_\downarrow) = \alpha (1 - 2 p_\uparrow)$$ (where $\alpha$ is some constant proportional to the magnetic field strength).

The entropy is zero for both extreme cases $p_\uparrow = 1$ and $p_\uparrow = 0$ and positive for all other values, while the average potential energy is $U = \alpha$ in the one case and $U = -\alpha$ in the other!

Entropy is a quantity arising when considering ensembles and is not in correspondence energy. Asking why we need entropy if we have potential energy is a bit like asking why we need Newtons equations of motion, if we can solve static problems with just the equations $\sum F_n = 0$ and $\sum \tau_n = 0$.

(Outlook: Temperature can be defined as $T(S, V, N) = \frac{1}{\partial_S E(S, V, N)}$ for the microcanonical ensemble, where $E$ is the energy of the system and $S$ its entropy, while $V$ and $N$ are volume and particle number).

Answer 6

This is an excellent question, which I think primarily emerges due to lazy science teaching.

The two principles you mention differ in their generality. Entropy maximization is - so far as we know - completely general. If you isolate a system from outside influences and leave it alone, it will transform into the state with the highest possible entropy that doesn't violate the conservation of energy. Note that the distinguishing feature of energy here is its conservation; it is not minimized, it is conserved (remains constant).

Principles of energy minimization however are special cases, where the maximization of entropy leads to a particular form of energy being minimized. Let's take the example of a ball dropped from a tall building. It's commonly asserted that the ball will minimize its gravitational potential energy, but this is only true because the ball loses kinetic energy to the air molecules in the atmosphere, and to the ground on impact. In the absence of an atmosphere, and with perfectly elastic collisions, the ball would bounce forever, perpetually exchanging potential for kinetic energy and vice versa. It's because of the dispersion of energy as heat (the maximization of entropy) that the portion of the ball's total energy we refer to as gravitational is minimized. Ultimately, the warmer ball at groundlevel is a higher entropy state than the cooler ball sitting on the rooftop.

As this example shows, the confusion is largely due to the way we partition a system's energy into different forms (e.g. gravitational, thermal etc), which locally are minimized. Energy in aggregate is conserved, but the quantity taking a particular form can be changed. It’s therefore important to specify which form of energy is being minimized in a particular case.

It has been suggested that confusions of this kind might be avoided if less emphasis were placed on the different 'forms' of energy in the science curriculum. An excellent case is made for this here: https://www.researchgate.net/publication/253046883_Energy_forms_or_energy_carriers

Answer 7

You are right in saying that a system want to maximize the entropy and minimize the potential energy, but these two tendencies are not one the opposite each other, it's not just simply energy = 1/entropy, they are independent but balanced in a certain way.

However, the usefulness of entropy (and of course, also of potential energy) strongly depends on the system you are interested in. In a purely mechanical system, with no dissipation, potential energy tells you everything you need, most of the times. But what if in your system there is an energy influx? Or what if there is dissipation? Or what if there are a lot of interacting particles?

Take another example, that of a system with constant volume and temperature, such as molecules in a box at a given temperature. In this case, the "effective energy" that the system wants to minimize is not the energy nor the entropy, but a mixture of the two things called Helmholtz free energy, written as:

$$F=U-TS$$

where $U$ is the potential energy and $S$ is the entropy. As you can see, the "role" of entropy is weighted with $T$, the temperature and has a minus sign in front of it.

This means that such a system want to minimize $F$ and can do it in two ways: minimize $U$ or maximize $S$ or find a balance of the two things so that in the end $F$ is small. How much entropy plays a role, is given by the temperature $T$. At very high temperatures, energy is less relevant because entropy is stronger, the flows of heat disrupt any order energy attempts to form. At $T=0$ entropy, on the other hand, does not play any role at all.

Now I am going to blow your mind: there are systems (with real life applications!) for which you can assume that $U=0$ i.e. they do not interact energetically. An example is the perfect gas. So how would such systems evolve? The answer is that they can use entropy as a driving force for finding their preferred state, so in this case it is potential energy which is irrelevant ($F=-TS$) and entropy dominates.

In this easy case (perfect gas) maximizing entropy leads to pure disorder, but there are cases (where $U!=0$ but very little relevant anyways) in which entropy is the force which leads to ordered structures.

You might want to check this out https://www.sif.it/riviste/sif/ncr/econtents/2019/042/11/article/0 for some examples (but it is not for beginners!)

So, answering your question, energy, entropy (but also entalpy, chemical potential ecc.ecc.) are just ways of describing the tendency that a system has in reaching its final state, given the conditions it is in. None of these ones is superior to the other ones, it just depends on the description you choose (again, for mechanics: energy. For chemistry: entropy and entalpy and chemical potential. ecc. ecc.)

Thus:

-entropy and energy are independent, you can not describe one as a function of the other one, they are yes bound, but not the same thing. They measure different things of the system, and their role is dependent on whether the thing they are measuring (energy measures interactions, entropy measures order, in a way) is the relevant one

-the description you choose (mechanics? chemistry? one particle or 1000 particles?) changes the role energy and entropy have

-entropy also has several other applications in computer science, ecc. ecc. (but that is not really really entropy).

Answer 8

The principle of minimum energy is essentially a restatement of the second law of thermodynamics.

Consider, for one, the familiar example of a marble on the edge of a bowl. If we consider the marble and bowl to be an isolated system, then when the marble drops, the potential energy will be converted to the kinetic energy of motion of the marble. Frictional forces will convert this kinetic energy to heat, and at equilibrium, the marble will be at rest at the bottom of the bowl, and the marble and the bowl will be at a slightly higher temperature. The total energy of the marble-bowl system will be unchanged. What was previously the potential energy of the marble, will now reside in the increased heat energy of the marble-bowl system. This will be an application of the maximum entropy principle as set forth in the principle of minimum potential energy, since due to the heating effects, the entropy has increased to the maximum value possible given the fixed energy of the system.

If, on the other hand, the marble is lowered very slowly to the bottom of the bowl, so slowly that no heating effects occur (i.e. reversibly), then the entropy of the marble and bowl will remain constant, and the potential energy of the marble will be transferred as energy to the surroundings. The surroundings will maximize its entropy given its newly acquired energy, which is equivalent to the energy having been transferred as heat. Since the potential energy of the system is now at a minimum with no increase in the energy due to heat of either the marble or the bowl, the total energy of the system is at a minimum. This is an application of the minimum energy principle.

Source: https://en.wikipedia.org/wiki/Principle_of_minimum_energy

Answer 9

Entropy is used in many different fields (classical thermodynamics, statistical thermodynamics, information theory, statistics) with very similar definitions, and those definitions fit well with one another.

So please tell me, if potential energy supposedly can serve all the purposes, how do you plan to use it for data compression?

Answer 10

Reaching a state of maximum entropy is NOT the same as running out of fuel, but the concepts are similar. It would make a good substitute for someone who really doesn't understand entropy (the same way voltage is described as "pressure").

Although I've never liked the idea of entropy going UP as we reach a state of greater homogenity and randomness. Maybe it should have been defined the opposite way; at least then we could have said going down to zero is maximum homogenity and randomness. (Perhaps we could call that antropy?)

Answer 11

In the following example, one can read about a case where the potential energy decrease is forced on a system, while the increase in the entropy of the system is a consequence hereof.

Imagine the particular situation of huge a collection of equal-mass, distinguishable particles. One-half of the particles possesses an electric charge, while the other half of the particles possesses an equal but opposite electric charge which means, of course, that the total charge of all particles is zero (so the number of particles is even, but that aside).

The particles find themselves in a box (through which electromagnetic radiation, or photons, can pass though, so maybe it's best to conduct this "experiment" in empty space where no e.m. radiation is present to enter the box; let's ignore the CMBR though I doubt it has a significant influence).

Let's assume the particles are initial in a configuration for which the potential energy is maximal after which we watch how the configuration develops. This configuration equals the configuration of atoms in a box, which has the highest entropy, which is true for the initial configuration if the particles had no charge. But our particles have electric charge...

From the beginning, the system develops towards a situation with higher entropy (such is the law). This law isn't the driving force though. You might think this maximum entropy already occurs in the begin configuration, but because the oppositely charged particles will tend to accelerate to each other, they acquire less potential energy (which implies the total kinetic energy of all particles increases) and are emitting e.m. radiation (photons, which contribute to the entropy). This development depends on the initial velocities of the particles (or on their initial momenta but as the masses of all particles are the same we can talk just as well about the velocities) which we consider as normally distributed for a maximal entropy (in the sense described in the preceding paragraph).

It's clear the e.m. forces drive the initial configuration to configurations with less and less electrical potential energy.

The e.m. force drives the configuration to configurations with less and less potential energy. At the same time, the entropy of the configuration increases (by the appearance of a huge number of photons that can escape out of the box). So the process is NOT driven by the law that the entropy of a closed system (which this system is, although photons can escape out of the box) always increases, but by the e.m. forces which decrease the potential energy with as a consequence that the entropy increases.