41
$\begingroup$

I was reading this article from Ethan Siegel and I got some doubts about a sentence about entropy, specifically when Ethan explains the irreversibility of the conditions of the hot-and-cold room, as in this figure:

entropy

In his words:

It's like taking a room with a divider down the middle, where one side is hot and the other is cold, removing the divider, and watching the gas molecules fly around. In the absence of any other inputs, the two halves of the room will mix and equilibrate, reaching the same temperature. No matter what you did to those particles, including reverse all of their momenta, they'd never reach the half-hot and half-cold state ever again.

My question is:

Is the spontaneous evolution from the equilibrium temperature (right side of the image) to the half-hot and half-cold state (left side) physically and theoretically impossible/forbidden, or is it simply so astronomically unlikely (from a statistical perspective) that in reality it never happens? The article seems to suggest the former, but I was under the impression of the latter.

$\endgroup$
  • 3
    $\begingroup$ I've removed a number of comments that were attempting to answer the question and/or responses to them. Commenters, please keep in mind that comments should be used for suggesting improvements and requesting clarification on the question, not for answering. $\endgroup$ – David Z Jul 19 at 2:16
  • 1
    $\begingroup$ Can doubt sometimes mean question? $\endgroup$ – Peter Mortensen Jul 19 at 21:15
  • $\begingroup$ With "doubt" I meant that I was feeling uncertain about my knowledge on the subject, given that the article provides an explanation that is in conflict with what I knew. $\endgroup$ – Andy4983948 Jul 19 at 22:31
  • 1
    $\begingroup$ The closure reasons once upon a time mentioned the idea that questions should not require book-length answers. As evidence that there is at least a chapter-length treatment of this see chapter 2 of Huw Price's Time's Arrow and Archimedes' Point, published in 1996 by Oxford University Press. $\endgroup$ – JdeBP Jul 20 at 7:05
  • $\begingroup$ I was once playing a game of pool and after the break all the spots ended up on one side of the table, and all the stripes on the other. It's an extremely unlikely outcome but (clearly) not impossible. Repeat as a thought experiment with more balls. The probability goes down very quickly but it's never 0. $\endgroup$ – Qwerky Jul 20 at 11:18
28
$\begingroup$

The appropriate mathematical tool to understand this kind of question, and more particularly Dale's and buddy's answers, is large deviation theory. To quote wikipedia, "large deviations theory concerns itself with the exponential decline of the probability measures of certain kinds of extreme or tail events". In this context, "exponential decline" means: probability that decreases exponentially fast with the increase of number of particles.
TL;DR: it can be shown that the probability to observe an evolution path for a system that decreases entropy is non-zero, and it decreases exponentially fast with the number of particles; thanks to a statistical mechanics of "trajectories", based on large deviation theory.

Equilibrium statistics

In equilibrium statistical mechanics, working in the appropriate thermodynamical ensemble, for instance the microcanonical ensemble in this case, one could relate the probability to observe a macrostate $M_N$ for the $N$ particles in the system, to the entropy of the macrostate $S[M_N]$: $\mathbf{P}_{eq}\left(M_N\right)\propto\text{e}^{N\frac{\mathcal{S}[M_N]}{k_{B}T}}.$ Naturally, the most probably observed macrostate, is the equilibrium state, the one which maximizes the entropy. And the probability to observe macrostates that are not the equilibrium state decreases exponentially fast as the number of particles goes to infinity, this is why we can see it as a large deviation result, in the large particle numbers limit.

Dynamical fluctuations

Using large deviation theory, we can extend this equilibrium point of view: based on the statistics of the macrostates, to a dynamical perspective based on the statistics of the trajectories. Let me explain.

In your case, you would expect to observe the macrostate of your system $(M_N(t))_{0\leq t\leq T}$, evolving on a time interval $[0,T]$ from an initial configuration $M_N(0)$ with entropy $S_0$ to a final configuration $M_N(T)$ with entropy $S_T$ such as $S_0 \leq S_T$, $S_T$ being the maximal entropy characterizing the equilibrium distribution, and the entropy of the macrostate at a time $t$, $S_t$ being a monotonous increasing function (H-Theorem for the kinetic theory of a dilute gas, for instance).

However, as long as the number of particles is finite (even if it is very large), it is possible to observe different evolutions, particularly if you wait for a very long time, assuming your system is ergodic for instance. By long, I mean large with respect to the number of particles. In particular, it has been recently established that one could formulate a dynamical large deviation result which characterizes the probability of any evolution path for the macrostate of the system (https://arxiv.org/abs/2002.10398). This result allows to evaluate for large but finite number of particles, the probability to observe any evolution path of the macrostate $(M_N(t))_{0\leq t\leq T}$, including evolution paths such as $S_t$, the entropy of the system a time $t$ is non monotonous. This probability will become exponentially small with the number of particles, and the most probable evolution, that increases entropy, will have an exponentially overwhelming probability as the number of particles goes to infinity.

Obviously, for a classical gas, N is very large, such evolution paths that do not increase entropy won't be observed: you would have to wait longer than the age of the universe to observe your system doing this. But one could imagine systems where we use statistical mechanics, where $N$ is large but not enough to "erase" dynamical fluctuations: biological systems, or astrophysical systems for instance, in which it is crucial to quantify fluctuations from the entropic fate.

| cite | improve this answer | |
$\endgroup$
  • 11
    $\begingroup$ So to sum up: the OP’s assumption is actually correct? There is nothing physically preventing such a reduction in entropy: it’s just so statistically improbable that you’d have to wait longer than the universe is old, right? $\endgroup$ – Chris Melville Jul 19 at 9:07
  • 1
    $\begingroup$ Exactly, there is nothing physically preventing a reduction in entropy. I guess the other important thing is there are systems in which the number of particles is still large, but smaller, where the "probability distribution" of evolution path is not as peaked around a path monotonously increasing the entropy. For instance, we use entropic tools to describe globular clusters, where the number of "particles" which are stars, is $N\sim 10^5$ ; in this case it is interesting to go beyond the most probable evolution in the study. $\endgroup$ – ErgodicRoller Jul 19 at 9:22
  • $\begingroup$ Interesting. Meaning, after the eventual heat death of the universe, given infinite time, the universe could suddenly re-appear at any point, in any state? :) $\endgroup$ – Chris Melville Jul 19 at 12:23
  • 1
    $\begingroup$ I am not really sure about that and that I am the most qualified to answer. I just took this opportunity to evoke large deviation theory, which is an elegant mathematical way to write statistical mechanics and to close the seemingly "irreversibility paradox"; I am familiar with stat-mech, but still as a student and I do not really know about cosmology. Concerning your question, to answer yes, one would have to assume some sort of ergodicity hypothesis about the whole universe, and I would not really know how to treat extra "classical mechanics" effect, such as chemical/nuclear reactions, etc. $\endgroup$ – ErgodicRoller Jul 19 at 12:36
  • 5
    $\begingroup$ It's called the Poincaré recurrence time $\endgroup$ – tusky_mcmammoth Jul 20 at 8:36
23
$\begingroup$

What you are interested in is Crook’s fluctuation theorem. It gives the probability of going “backwards” thermodynamically. Specifically, the theorem says:

$$\frac{P(A\rightarrow B)}{P(A\leftarrow B)}=\exp \left( \frac{1}{k_B T}(W_{A\rightarrow B}-\Delta F) \right)$$

In the case of the box, $W_{A\rightarrow B}=0$ so the probability is purely driven by the change in Helmholtz free energy, $\Delta F$.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Crook's theorem is one of the most beautiful things I saw in thermodynamics. It's quite amazing it took until mid 90' for people to discover it. $\endgroup$ – AlanR Jul 21 at 18:17
7
$\begingroup$

Noticing that Shannon information entropy is related to thermodynamic entropy like this:

$$ S = k_B H $$

One can express the quantum entropic uncertainty principle for thermodynamic entropies:

$$ S_a + S_b\geq k_B\log\left(\frac e2\right) $$

Where $S_a, S_b$ is temporal and spectral thermodynamic entropies. This shows that entropies can fluctuate in time and spectra. It's not forbidden for entropy fluctuation going backwards, but likely this will be on short time scales and within small partitions of the whole system. And probably backwards entropy fluctuations will be canceled later some time by standard time arrow fluctuations. So not much useful information can be extracted from backwards fluctuations because in principle they are uncontrolable.

Also Bohr suggested a thermodynamic uncertainty relation: $$ {\mathrm{\Delta }}\beta \ge \frac{1}{{{\mathrm{\Delta }}U}} $$

Where $\beta = (k_BT)^{-1}$ is inverse temperature. This relationship means that if you know the system internal energy very precisely, then you don't know anything about its temperature and vise-versa. Now imagine that after molecules diffusion in part A you measure the temperature exactly and the exact internal energy of the B part. Then according to the uncertainty principle it can be that this measurement resulted in half-hot / half-cold molecule partition formation. But, this implies that the measurement has performed some kind of thermodynamic work, so this has nothing to do with spontaneous backwards entropy change and thus falls out of the question formulated by the OP. But still I think it's interesting to think about such kind of possibility, because the act of measurement is vaguely defined and may happen without human intervention.

| cite | improve this answer | |
$\endgroup$
6
$\begingroup$

Well, there was a thought experiment by Maxwell (known as Maxwell's Demon), in which if one knows about exact information of all the particles in both the compartment then he/she cant timely open the partition so as to let the particle(s) with high energy on one side and leave the particles with low energies on the other. Now doing it all and having exact information about all the particles is next to impossible, let's assume if one could do so it won't be spontaneous.

Now talking about the probability of that event happening, imagine you flip a coin 10000 times what do you expect regarding the result ie. number of tails vs no. of heads, as law of large no. states it will be close to 50-50 so it's highly unlikely that you'll get 9999 heads and a tale.

Returning to you question there are molecules of the order $10^{26}$ for a just a mole of gas and with that amount of molecules, for the molecules to separate you need only one kind of the particle to pass through the partition hence you can think of how unlikely the event is when you cannot get 9999 tails from just 10000 flips (the coin experiment is just an analogy you can assume that a tails is a particle with high energy and heads a particle with low energy or vice versa going through the partition, also I have assumed the fact that collisions didn't occur to keep their velocities as same as before which is also impossible).

So yes it is astronomically unlikely.

| cite | improve this answer | |
$\endgroup$
  • 33
    $\begingroup$ I don't think that "theoretically" means what you think it means. $\endgroup$ – WillO Jul 18 at 19:46
  • 1
    $\begingroup$ If it were theoretically impossible, we wouldn't expect those things to eventually happen (on a macroscopic scale) when watching the "heat death" in the far future of the Universe. A Boltzmann brain would be even more impossible. $\endgroup$ – Rodrigo Amaral Jul 19 at 3:36
  • $\begingroup$ @WillO well I agree with you, I know that the system could have a state like that for a moment just that the probability is extremely small, but what I meant was that for the temperature change to be observed that state won't contribute much as the next moment it would change, so it's like the Boltzmann brain that may form and deteriorate without being observed. So I think that original state could never be restored which wasn't dynamic on a macroscopic scale. Please correct me if I got this this wrong. $\endgroup$ – buddy001 Jul 19 at 8:00
  • $\begingroup$ Hi Buddy001, I think this isn't a bad answer, except for the last sentence which is controversial and makes it look wrong. If there were 10 particles, it wouldn't be impossible. Nor 20 - but at 10^26, you say it is impossible, and that there is a theory that predicts it. If you keep the last line, I think you should say what that theory is, and at what value is the predicted breakover point. Otherwise, that line just sounds like hyperbole, which I don't think fits a hard-science site like this. $\endgroup$ – SusanW Jul 20 at 17:20
  • $\begingroup$ I'm pretty sure you mean "heads and tails" but I'm not 100% confident it's not a regional difference so I didn't edit it directly. $\endgroup$ – stevenjackson121 Jul 20 at 18:13
3
$\begingroup$

Is the spontaneous evolution from the equilibrium temperature (right side of the image) to the half-hot and half-cold state (left side) physically and theoretically impossible/forbidden,

No.

or is it simply so astronomically unlikely (from a statistical perspective) that in reality it never happens?

Yes.

I'll extend my terse answer but don't want to go to long because frankly I don't think a long answer is needed for this question. I do not understand why physicists wring their hands so badly about this. Start with the atoms as they are in the picture on the left and remove the divider. Let the system evolve for 10 minutes. By our usual definition of entropy (related to the number of red and blue particles on each side) the system will basically have maximal entropy. Take a snapshot of the exact position and momentum of each particle.

Now, start over with the exact same number of particles. Place them in the exact positions needed, at at the start of the experiment give them a kick of momentum so they have the exact same momentum as they had at the end of the previous experiment. Newton's laws are reversible. This means the particles WILL go back to the configuration of all red on one side and all blue on the other side.

There should be absolutely nothing controversial about this. The initial state I described for the second experiment is a perfectly valid state within configuration space. Theoretically I'm allowed to specify ANY position and momentum that I like for all particles. Newton's laws are reversible. Period. This is explain my "No." answer to the OPs first question.

So that is the theoretical part of the answer. Now, the practical part of the answer. Why don't we ever see this happen? Well that has been answered in many words by all of the other answers here. The reason is that it is unbelievably unlikely. Calling it astronomically unlikely GREATLY overstates the magnitude of astronomical scales. This explains the "yes." answer to the OPs second question.

Now a little bonus that wasn't addressed by my answer yet: One way to the think about the 2nd law of thermodynamics is this. The entropy of a state tells you how statistically probable it would be to find the system in this state. The second law of thermodynamics says that over time it is HIGHLY likely that, compared to the state a system is in now, the state the system is in in the future is going to be a state that it is more statistically probable to find the system in. More sharply: "We are more likely to find a system in states which we are more likely to find a system in."

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ From my understanding of Landauer's principle, the initial state of the second experiment has zero entropy. Because its microscopic state is known, all its heat can be theoretically extracted as work. The process of "taking a snapshot" can be thought of as a generalised form of Maxwell's demon, where measurements merely shifts entropy to the demon in the gas–demon system. $\endgroup$ – timuzhti Jul 21 at 2:22
  • $\begingroup$ @timuzhti There's nothing stopping that state from arising naturally from some random initial conditions. No Maxwell's demon required. Is that likely to happen? No. Is it forbidden by the laws of physics? Absolutely not. I don't see what is complicated or subtle about this point. $\endgroup$ – jgerber Jul 21 at 2:38
  • $\begingroup$ That doesn't change the fact that any specific microstate at the start of the experiment has no entropy. If we take a snapshot of all the air molecules in your room isentropically and then find another room where the air molecules were magically in the exact same state, the air in that room would have no entropy either. $\endgroup$ – timuzhti Jul 21 at 15:30
  • $\begingroup$ This is the hand wringing I'm talking about. What you're saying doesn't make sense. Forget about the experimentalists preparing the state like I talked about in my question. Just imagine the state I described at the start of the 2nd experiment. Image you look at a random setup of 2 boxes like described. You just walked in from outside, the boxes are natural in a 300K room. Any reasonable physicist would describe the boxes as having high entropy. 10 minutes later the 2-box system evolves to the state described. another 10 minutes pass and all of the red particles go to one side and blue $\endgroup$ – jgerber Jul 21 at 15:53
  • 1
    $\begingroup$ I like your bonus answer. It reminds me of the principle of natural selection: Survival of the fittest because the fittest survive. $\endgroup$ – Neal Jul 22 at 2:45
3
$\begingroup$

Entropy is the measure of how spread out energy is compared to the maximum amount it could be spread out. The mathematics show that the predicted increase in the entropy of the universe (the second law of thermodynamics) is a result of the statistical probability that energy will trend toward a more spread out (vs. concentrated) state.

Although this process seems irreversible, statistically it is also inevitable, over a long enough time span, that the energy of universe will, by the same probability-based reasoning, redistribute to a minimum energy configuration (or most highly concentrated state). This probability is so low it is almost impossible to describe except to say that it is not infinitely unlikely, and therefore eventually it will occur.

Interestingly, one of the greatest living physicist, Roger Penrose, has argued that there is a huge mystery in cosmology related to entropy, namely that there is no explanation for how the initial very-low entropy state of the universe could have occurred.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

Poincare recurrence has been mentioned in a comment by tusky_mcmammoth, but I think it's worth highlighting as an answer to illustrate both a piece of interesting mathematics and a limit of mathematical modeling.

A mathematical model of "particles in a box" treats the particles as points that elastically collide with each other and the container. Because the particles are confined and energy is conserved, the Poincare recurrence theorem actually guarantees that the system will with eventually return arbitrarily closely to its initial conditions!

Of course, in reality the universe will freeze to death first. The time this takes is enormous. (For example, this paper numerically computes Poincare recurrence times for completely integrable systems using some tricks from number theory.)

One could paraphrase the story of the butterfly and the diamond mountain to say:

There is a diamond mountain. Once every thousand years, a butterfly visits it and touches it once. By the time the butterfly has worn the mountain down to nothing, a complex system's Poincare recurrence time has just begun to elapse.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I've seen plausible-looking objections somewhere about Poincare recurrence having some uncertainty due to the universe being spherical in form (rather than having the box shape specified by Poincare) and pi having an indefinitely variable number of digits. Maybe someone could shoot this down so I can sleep tonight. $\endgroup$ – Edouard Jul 21 at 19:25
  • $\begingroup$ A closer look at the answer, as well as a look at the Wiki "Poincarre recurrence", shows that uncertainty or imperfection remain a disappointing part of the picture (and that the actual theorem wasn't Poincarre's,altho apparently inspired by remarks of his), so I'm upvoting this answer, which (as usual) doesn't necessarily exclude the plausibility of the others. Lack of imperfection might not characterize life, anyway. $\endgroup$ – Edouard Jul 22 at 2:03
  • 1
    $\begingroup$ @Edouard The key here is that Poincare recurrence is a property of a specific class of mathematical dynamical systems, not necessarily of nature. Because these mathematical models are idealized, they approximate the real system for a period of time that is much much less than the time it takes for Poincare recurrence to occur. (I am not sure what the shape of the universe and the digits of $\pi$ have to do with this, although last I heard the universe's curvature is indistinguishable from zero.) $\endgroup$ – Neal Jul 22 at 2:41
  • $\begingroup$ Yes, the spacetime curvature that results in gravity is, I believe even in "localities" much larger than our observable region, often described as negligible. Dr. Norton of the University of Pittsburgh has a website on General Relativity in which he mentions that much more of the curvature is temporal than spatial, but, although that notion seems intuitively correct (given the length of time during which our species observed both the difference between day & night and the terrestrial horizon without considering the earth's curvature to be complete), I'm unsure why it does. $\endgroup$ – Edouard Jul 22 at 13:27
1
$\begingroup$

While nothing's been proven, current theories posit that a black hole's entropy changes in inverse proportion to its mass/energy: i.e., when it decays, its entropy increases. Most black holes spend most of their early lifespans increasing in mass, and they would be decreasing in entropy during this time.

Now this isn't a net loss of entropy: the release of energy black holes produce ripping apart matter and--more than likely--spacetime leads to the inevitable net increase in entropy our favorite law of thermodynamics requires.

In the context of just the black hole and the matter it's vacuuming up: yes, entropy spontaneously decreases. But unless our entire universe were to be contained in a black hole, even these cosmological titans still produce a net increase of entropy.

| cite | improve this answer | |
$\endgroup$
-4
$\begingroup$

Decreasing entropy seems impossible, not just improbable.

Here's why:

  1. I am using the formula: $\Omega = \frac{N!}{j!*(N-j)!}$ where $\Omega$ is the number of microstates (probability) of having j and (N-j) particles in each half of an isolated system (box), N is the number of particles in the system and j is the number of particles in one half while (N-j) is the number of particles in the other half. With this formula, I am getting the probability of a 2% fluctuation $\frac{(N/2 - j)}{N/2}$ of 98% for N=200, 10% for N=1000, 0.1% for N=100,000 and one in 10^21 for N=1,000,000. Therefore, no decreasing entropy will ever be observed in a macro-system.
  2. Given ~10^25 molecules in a cubic meter, isolated microsystems of 1000 molecules are NOT realistic. Why assume they would behave like their macrosystem equivalents when just about anything has different properties at such small comparative scale? Let us NOT assume 1000 molecules behave just like 10^25 molecules.
  3. Arrow of time is given by entropy increase (think movie of an egg breaking). Since no one has ever witnessed the reversibility of the arrow of time, why assume such thing?
  4. The model used for systems of particles is demonstrably false. Pebbles on a Go board is NOT how gases behave. Pebbles stay put whereas compressed gases push back forcefully.
| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.