We said $T=E/N$ is our measure of the average energy in any degree of freedom and now we see that as energy $\delta E$ leaves our bouncing ball, it causes an entropy increase of about $\delta s=\delta E/T$. So the minimum energy principle is a special case of the maximum entropy principle. The limit holds when the energy in the system is much larger than the temperature of a big reservoir that it can share energy with, $U \gg E/N$ but itthis does not change the temperature totoo much as it shares this energy with the systems.system releases its energy, $U \ll E.$
We said $T=E/N$ is our measure of the average energy in any degree of freedom and now we see that as energy $\delta E$ leaves our bouncing ball, it causes an entropy increase of about $\delta s=\delta E/T$. So the minimum energy principle is a special case of the maximum entropy principle when the energy in the system is much larger than the temperature of a big reservoir that it can share energy with, but it does not change the temperature to much as it shares this energy with the systems.
We said $T=E/N$ is our measure of the average energy in any degree of freedom and now we see that as energy $\delta E$ leaves our bouncing ball, it causes an entropy increase of about $\delta s=\delta E/T$. So the minimum energy principle is a special case of the maximum entropy principle. The limit holds when the energy in the system is much larger than the temperature of a big reservoir that it can share energy with, $U \gg E/N$ but this does not change the temperature too much as the system releases its energy, $U \ll E.$
So if we have $E$ lumps of energy[Note 2] that we can spread across $N$ degrees of freedom, we can describe the random distribution as a bit-string with $E$ zeroes representing units of energy and $N-1$ ones representing partitions between degrees of freedom, so 001000100110
means 2 lumps in the first box, 3 in the second, 2 in the third, 0 in the fourth, 1 in the fifth. The number of ways to arrange the system is then given by combinatorics as $$W=\frac{(E+N-1)!}{E!(N-1)!}$$$$W=\frac{(E+N-1)!}{E!(N-1)!}.$$
andSo now imagine your situation: you have a couple of degrees of freedom that you can see directly: say, the kinetic and potential energy of a bouncing basketball. They have some energy which I will just call $U$. And then you have the degrees of freedom that you cannot see directly, all of the vibration modes of the floor and the sound in the air, which I describe with the above system and its energy $E$. And we have energy conservation: $U + E$ is constant. So as energy leaves the system $U$ and hence leaves our certainty, and flows into this environment, it increases this $E$. This last
Now if we compare $W(E)$ and $W(E-1)$ we will find that the most recent lump multipliedof energy flowing from $U$ into $E$, had the effect of multiplying $W$ by $(E+N-1)/E\approx 1 + N/E$$W(E) = W(E-1)\cdot(E+N-1)/E.$ We can approximate this for usual systems[Note 3] as $\approx 1 + N/E$ and we measure entropy with the logarithm of $W$, so it added to $s=\ln W$ an amount $\ln(1+N/E)\approx N/E$, assuming that the typical degree of freedom has many lumps of energy on average.
Notes
Note 1: the rough height of the atmosphere can be estimated by assuming it all has the same density $\rho$ as it does at the ground, in which case a column of air of that density exerts a pressure $P = \rho~g~h$, so that $h = P/(\rho~g)$, which using the density of nitrogen gas at Earth's surface gives h ≈ 8 km, which is roughly correct; the troposphere ends maybe 9-17 km in the sky, and the vast majority of the air is in the troposphere—so we are only off by a factor of 2. If you were instead to do the calculation assuming the nitrogen gas became liquid nitrogen, you would find that the figure is only h ≈ 13 m, much thinner, before low-temperature attractions start to take over. Intuitively, this effect called temperature is the big difference between liquid and gaseous nitrogen. But there is another way to calculate this number of ~8 km that really seals the deal: remember that temperature is supposed to be the average energy in every degree of freedom? Well, that energy $k_\text B T$ for room temperature is typically about $k_\text B T\approx25~\text{meV}$, milli-electron-volts, where an electron-volt is the energy an electron gains if it moves through a potential difference in vacuum of 1 volt. The mass of a nitrogen atom is about $m=28~\text{amu}$, and therefore the average energy in the height degree of freedom should be $k_\text B T = m~g~h$, giving a sort of average height... of h ≈ 9 km.
Note 2: That Boltzmann’s theory of entropy requires us to consider things like energy to come in lumps, is something that caused a great deal of difficulty in the early development of the theory. Then a theory called quantum mechanics came around and it said that for real systems, energy really does come in lumps, saving the theory. Before that happened, in 1905, a young and then-unknown Albert Einstein pointed out that it really does matter how big the lumps are, by noticing that if there are lots of little lumps then their fluctuations tend to average out, but if there are a few big lumps then their fluctuations are more noticeable. During this he also provided a first inkling of a truly powerful result in this statistical physics called the fluctuation-dissipation theorem, which basically says that we can connect the rate at which this bouncing-ball is losing energy to its environment (its dissipation) with the amount of thermal noise it starts to have from the energy coming randomly back from the environment the other way (its fluctuation).
Note 3: just to give a rough scale of what we are talking about here, you might imagine that for a real system we have $N\approx 10^{23}$ given by the approximate scale of Avogadro’s number, while $E/N$ might be in the range of $10^6$ assuming a phonon frequency in the MHz, and the energy in your half-kg basketball might be $U\approx 10^{21}$ of these units. So in case it is not clear, the impact of absorbing all of that basketball kinetic energy on the temperature of the floor is approximately negligible and the thermal vibrations of the basketball are on the order of, whatever, $10^{-15}\text{ m}$ or a millionth of a nanometer fluctuation in its height: you would never notice such a thing.
So if we have $E$ lumps of energy that we can spread across $N$ degrees of freedom, we can describe the random distribution as a bit-string with $E$ zeroes representing units of energy and $N-1$ ones representing partitions between degrees of freedom, so 001000100110
means 2 lumps in the first box, 3 in the second, 2 in the third, 0 in the fourth, 1 in the fifth. The number of ways to arrange the system is then given by combinatorics as $$W=\frac{(E+N-1)!}{E!(N-1)!}$$
and as energy leaves the system and our certainty, and flows into this environment, it increases $E$. This last lump multiplied $W$ by $(E+N-1)/E\approx 1 + N/E$ and we
measure entropy with the logarithm of $W$, so it added to $s=\ln W$ an amount $\ln(1+N/E)\approx N/E$.
Note 1: the rough height of the atmosphere can be estimated by assuming it all has the same density $\rho$ as it does at the ground, in which case a column of air of that density exerts a pressure $P = \rho~g~h$, so that $h = P/(\rho~g)$, which using the density of nitrogen gas at Earth's surface gives h ≈ 8 km, which is roughly correct; the troposphere ends maybe 9-17 km in the sky, and the vast majority of the air is in the troposphere—so we are only off by a factor of 2. If you were instead to do the calculation assuming the nitrogen gas became liquid nitrogen, you would find that the figure is only h ≈ 13 m, much thinner, before low-temperature attractions start to take over. Intuitively, this effect called temperature is the big difference between liquid and gaseous nitrogen. But there is another way to calculate this number of ~8 km that really seals the deal: remember that temperature is supposed to be the average energy in every degree of freedom? Well, that energy $k_\text B T$ for room temperature is typically about $k_\text B T\approx25~\text{meV}$, milli-electron-volts, where an electron-volt is the energy an electron gains if it moves through a potential difference in vacuum of 1 volt. The mass of a nitrogen atom is about $m=28~\text{amu}$, and therefore the average energy in the height degree of freedom should be $k_\text B T = m~g~h$, giving a sort of average height... of h ≈ 9 km.
So if we have $E$ lumps of energy[Note 2] that we can spread across $N$ degrees of freedom, we can describe the random distribution as a bit-string with $E$ zeroes representing units of energy and $N-1$ ones representing partitions between degrees of freedom, so 001000100110
means 2 lumps in the first box, 3 in the second, 2 in the third, 0 in the fourth, 1 in the fifth. The number of ways to arrange the system is then given by combinatorics as $$W=\frac{(E+N-1)!}{E!(N-1)!}.$$
So now imagine your situation: you have a couple of degrees of freedom that you can see directly: say, the kinetic and potential energy of a bouncing basketball. They have some energy which I will just call $U$. And then you have the degrees of freedom that you cannot see directly, all of the vibration modes of the floor and the sound in the air, which I describe with the above system and its energy $E$. And we have energy conservation: $U + E$ is constant. So as energy leaves the system $U$ and hence leaves our certainty, and flows into this environment, it increases this $E$.
Now if we compare $W(E)$ and $W(E-1)$ we will find that the most recent lump of energy flowing from $U$ into $E$, had the effect of multiplying $W$ by $W(E) = W(E-1)\cdot(E+N-1)/E.$ We can approximate this for usual systems[Note 3] as $\approx 1 + N/E$ and we measure entropy with the logarithm of $W$, so it added to $s=\ln W$ an amount $\ln(1+N/E)\approx N/E$, assuming that the typical degree of freedom has many lumps of energy on average.
Notes
Note 1: the rough height of the atmosphere can be estimated by assuming it all has the same density $\rho$ as it does at the ground, in which case a column of air of that density exerts a pressure $P = \rho~g~h$, so that $h = P/(\rho~g)$, which using the density of nitrogen gas at Earth's surface gives h ≈ 8 km, which is roughly correct; the troposphere ends maybe 9-17 km in the sky, and the vast majority of the air is in the troposphere—so we are only off by a factor of 2. If you were instead to do the calculation assuming the nitrogen gas became liquid nitrogen, you would find that the figure is only h ≈ 13 m, much thinner, before low-temperature attractions start to take over. Intuitively, this effect called temperature is the big difference between liquid and gaseous nitrogen. But there is another way to calculate this number of ~8 km that really seals the deal: remember that temperature is supposed to be the average energy in every degree of freedom? Well, that energy $k_\text B T$ for room temperature is typically about $k_\text B T\approx25~\text{meV}$, milli-electron-volts, where an electron-volt is the energy an electron gains if it moves through a potential difference in vacuum of 1 volt. The mass of a nitrogen atom is about $m=28~\text{amu}$, and therefore the average energy in the height degree of freedom should be $k_\text B T = m~g~h$, giving a sort of average height... of h ≈ 9 km.
Note 2: That Boltzmann’s theory of entropy requires us to consider things like energy to come in lumps, is something that caused a great deal of difficulty in the early development of the theory. Then a theory called quantum mechanics came around and it said that for real systems, energy really does come in lumps, saving the theory. Before that happened, in 1905, a young and then-unknown Albert Einstein pointed out that it really does matter how big the lumps are, by noticing that if there are lots of little lumps then their fluctuations tend to average out, but if there are a few big lumps then their fluctuations are more noticeable. During this he also provided a first inkling of a truly powerful result in this statistical physics called the fluctuation-dissipation theorem, which basically says that we can connect the rate at which this bouncing-ball is losing energy to its environment (its dissipation) with the amount of thermal noise it starts to have from the energy coming randomly back from the environment the other way (its fluctuation).
Note 3: just to give a rough scale of what we are talking about here, you might imagine that for a real system we have $N\approx 10^{23}$ given by the approximate scale of Avogadro’s number, while $E/N$ might be in the range of $10^6$ assuming a phonon frequency in the MHz, and the energy in your half-kg basketball might be $U\approx 10^{21}$ of these units. So in case it is not clear, the impact of absorbing all of that basketball kinetic energy on the temperature of the floor is approximately negligible and the thermal vibrations of the basketball are on the order of, whatever, $10^{-15}\text{ m}$ or a millionth of a nanometer fluctuation in its height: you would never notice such a thing.
This is a great questionI love that you are asking about how these two similar principles relate. The answer isHowever, you assume that we can derive entropy maximizing from energy minimizing, and I don't think you can: I would instead derive the minimum energy principle from maximizing entropy. In my view, the minimum energy principle is fundamentally aboutan entropy. If result and until you don't understand entropy, then you don'twon’t understand the minimum energy principle. TheSo my explanation goesflows in the opposite direction of how you are understandingunderstand it.
Why not just energy?
To put it in harsh terms with an example, the minimum energy principle suggests that all the air molecules in the atmosphere should fall to the floor and stay there.[Note 1] That is the minimum of gravitational potential energy, and they have certainly had years to fall. Why aren't they there yet? Clearly there iswe should look for a more general principle than the minimum energy principle, andof which the minimum energy principle is a special case. The air staying in the atmosphere would satisfy the broader principle, but not the minimum energy principle.
The broaderIn softer terms with a constraint, isthe minimum energy conservationprinciple must not make ultimate sense of the world due to the constraint that energy is conserved. Energy can't be created or destroyed, but only spread around various “degrees of freedom” where it can live. This already suggestsuggests something wronglimiting about the minimum energy principle, because if you look at a big enough system the total energy must be constant: and there is no minimizing a constant. What is it about the energy of the basketball bouncing across the court that wants the energy to leave it? And why doesn't energy come back into it? One can point to the fact that friction does negative work, but one runs into the problem that this energy has to go somewhere.
How entropy solves this problem
The principle of maximum entropy says that we become more uncertain about the state of the whole world over time. If we don't know some number exactly, and that number affects other numbers in the dynamical evolution of the system, then we soon don't know those other numbers exactly: and so our uncertainties tend to multiply across the system. Two particles with uncertain momenta collide, their momenta afterwards are even less certain. Our idea about where the energy in a system is, becomes less and less accurate. Eventually, the energy is just randomly permuted across all of the degrees of freedom, and in the average it is roughly spread evenly across all the places it can be (in specific instances we may see a little more of it here, less there, but not terribly by much).
We call that state thermal equilibrium. We can measure something called the temperature by that average energy in each degree of freedom. All of that energy in all those degrees of freedom is then called thermal energy. We can typically only get at it by connecting two systems with different temperatures, and catching some of the energy as it flows from the hotter one to the colder one.
The minimum energy principle is, then, a specific case. It is the specific case of a “system” of a few well-defined degrees of freedom that hold a lot more energy than the temperature of an “environment” of a lot of other degrees of freedom, with which the system can share energy. In such a case, as we become more uncertain about the location of the energy in the world, more and more of it should be in the environment and less in the system, until the system is in thermal equilibrium with its environment and only has the thermal energy fluctuationsof that we were measuring as temperaturedegree of freedom. That temperature will increase as this energy is absorbed, but if there are lots of degrees of freedom in the environment, it doesn't have to increase by much. That just depends on the situation. For a large object like a soccer ball rolling down a hill, those thermal jitters can easily pass beneath observation.
So if we have $E$ lumps of energy that we can spread across $N$ degrees of freedom, we can describe the random distribution as a bit-string with $E$ zeroes representing units of energy and $N-1$ ones representing partitions between degrees of freedom, so 001000100110
means 2 lumps in the first box, 3 in the second, 2 in the third, 0 in the fourth, 1 in the fifth. The number of ways to arrange the system is then given by combinatorics as $$W=\frac{(E+N-1)!}{E!(N-1)!}$$
and as energy leaves the system and our certainty, and flows into this environment, it increases $E$. This last lump multiplied $W$ by $(E+N-1)/E\approx 1 + N/E$ and we
measure entropy with the logarithm of $W$, so it added to $s=\ln W$ an amount $\ln(1+N/E)\approx N/E$.
We said $T=E/N$ is our measure of the average energy in any degree of freedom and now we see that as energy $\delta E$ leaves our bouncing ball, it causes an entropy increase of about $\delta s=\delta E/T$. So the minimum energy principle is a special case of the maximum entropy principle when the energy in the system is much larger than the temperature of a big reservoir that it can share energy with, but it does not change the temperature to much as it shares this energy with the systems.
This is a great question. The answer is that the minimum energy principle is fundamentally about entropy. If you don't understand entropy, then you don't understand the minimum energy principle. The explanation goes the opposite of how you are understanding it.
To put it in harsh terms, the minimum energy principle suggests that all the air molecules in the atmosphere should fall to the floor and stay there.[Note 1] That is the minimum of potential energy, and they have certainly had years to fall. Why aren't they there yet? Clearly there is a more general principle the minimum energy principle, and the minimum energy principle is a special case.
The broader constraint, is energy conservation. Energy can't be created or destroyed, but only spread around various “degrees of freedom” where it can live. This already suggest something wrong about the minimum energy principle, because if you look at a big enough system the total energy must be constant: there is no minimizing a constant.
The principle of maximum entropy says that we become more uncertain about the state of the whole world over time. If we don't know some number, and that number affects other numbers in the dynamical evolution of the system, then we don't know those other numbers: and our uncertainties tend to multiply. Two particles with uncertain momenta collide, their momenta afterwards are even less certain. Our idea about where the energy in a system is, becomes less and less accurate. Eventually, the energy is just randomly permuted across all of the degrees of freedom, and in the average it is roughly spread evenly across all the places it can be (in specific instances we may see a little more of it here, less there, but not terribly by much).
We call that state thermal equilibrium. We can measure something called the temperature by that average energy in each degree of freedom.
The minimum energy principle is, then, a specific case. It is the specific case of a “system” of a few well-defined degrees of freedom that hold a lot more energy than the temperature of an “environment” of a lot of other degrees of freedom, with which the system can share energy. In such a case, as we become more uncertain about the location of the energy in the world, more and more of it should be in the environment and less in the system, until the system is in thermal equilibrium with its environment and only has the thermal energy fluctuations that we were measuring as temperature. That temperature will increase as this energy is absorbed, but if there are lots of degrees of freedom in the environment, it doesn't have to increase by much. That just depends on the situation. For a large object like a soccer ball rolling down a hill, those thermal jitters can easily pass beneath observation.
I love that you are asking about how these two similar principles relate. However, you assume that we can derive entropy maximizing from energy minimizing, and I don't think you can: I would instead derive the minimum energy principle from maximizing entropy. In my view, the minimum energy principle is fundamentally an entropy result and until you understand entropy, you won’t understand the minimum energy principle. So my explanation flows in the opposite direction of how you understand it.
Why not just energy?
To put it in harsh terms with an example, the minimum energy principle suggests that all the air molecules in the atmosphere should fall to the floor and stay there.[Note 1] That is the minimum of gravitational potential energy, and they have certainly had years to fall. Why aren't they there yet? Clearly we should look for a more general principle than the minimum energy principle, of which the minimum energy principle is a special case. The air staying in the atmosphere would satisfy the broader principle, but not the minimum energy principle.
In softer terms with a constraint, the minimum energy principle must not make ultimate sense of the world due to the constraint that energy is conserved. Energy can't be created or destroyed, but only spread around various “degrees of freedom” where it can live. This already suggests something limiting about the minimum energy principle, because if you look at a big enough system the total energy must be constant: and there is no minimizing a constant. What is it about the energy of the basketball bouncing across the court that wants the energy to leave it? And why doesn't energy come back into it? One can point to the fact that friction does negative work, but one runs into the problem that this energy has to go somewhere.
How entropy solves this problem
The principle of maximum entropy says that we become more uncertain about the state of the whole world over time. If we don't know some number exactly, and that number affects other numbers in the dynamical evolution of the system, then we soon don't know those other numbers exactly: and so our uncertainties tend to multiply across the system. Two particles with uncertain momenta collide, their momenta afterwards are even less certain. Our idea about where the energy in a system is, becomes less and less accurate. Eventually, the energy is just randomly permuted across all of the degrees of freedom, and in the average it is roughly spread evenly across all the places it can be (in specific instances we may see a little more of it here, less there, but not terribly by much).
We call that state thermal equilibrium. We can measure something called the temperature by that average energy in each degree of freedom. All of that energy in all those degrees of freedom is then called thermal energy. We can typically only get at it by connecting two systems with different temperatures, and catching some of the energy as it flows from the hotter one to the colder one.
The minimum energy principle is, then, a specific case. It is the specific case of a “system” of a few well-defined degrees of freedom that hold a lot more energy than the temperature of an “environment” of a lot of other degrees of freedom, with which the system can share energy. In such a case, as we become more uncertain about the location of the energy in the world, more and more of it should be in the environment and less in the system, until the system is in thermal equilibrium with its environment and only has the thermal energy of that degree of freedom. That temperature will increase as this energy is absorbed, but if there are lots of degrees of freedom in the environment, it doesn't have to increase by much. That just depends on the situation. For a large object like a soccer ball rolling down a hill, those thermal jitters can easily pass beneath observation.
So if we have $E$ lumps of energy that we can spread across $N$ degrees of freedom, we can describe the random distribution as a bit-string with $E$ zeroes representing units of energy and $N-1$ ones representing partitions between degrees of freedom, so 001000100110
means 2 lumps in the first box, 3 in the second, 2 in the third, 0 in the fourth, 1 in the fifth. The number of ways to arrange the system is then given by combinatorics as $$W=\frac{(E+N-1)!}{E!(N-1)!}$$
and as energy leaves the system and our certainty, and flows into this environment, it increases $E$. This last lump multiplied $W$ by $(E+N-1)/E\approx 1 + N/E$ and we
measure entropy with the logarithm of $W$, so it added to $s=\ln W$ an amount $\ln(1+N/E)\approx N/E$.
We said $T=E/N$ is our measure of the average energy in any degree of freedom and now we see that as energy $\delta E$ leaves our bouncing ball, it causes an entropy increase of about $\delta s=\delta E/T$. So the minimum energy principle is a special case of the maximum entropy principle when the energy in the system is much larger than the temperature of a big reservoir that it can share energy with, but it does not change the temperature to much as it shares this energy with the systems.