How do you prove the second law of thermodynamics from statistical mechanics?

How do you prove the second law of thermodynamics from statistical mechanics? To prove entropy will only increase with time? How to prove? Please guide.

• This is nearly a duplicate of this question: physics.stackexchange.com/q/20401 – Ben Crowell Oct 20 '13 at 19:47
• Also very near duplicate of physics.stackexchange.com/q/10690 and related to physics.stackexchange.com/q/63416 – joshphysics Oct 21 '13 at 1:55
• @outrageous Sorry to bring up an old thread but you and future readers might find this paper interesting. I am by no means an expert in this area so given below discussions it would be right to be weary. Statistical mechanical proof of the second law of thermodynamics based on volume entropy - Michele Campisi [link] sciencedirect.com/science/article/pii/S1355219807000974 – user58536 Jun 27 '15 at 10:39
• @BenCrowell Please see above link. Does it contrast main scientific thinking and the paper you linked? – user58536 Jun 27 '15 at 10:43

How do you prove the second law of thermodynamics from statistical mechanics?

You can't. In order to prove the second law, which is time-asymmetric, you need some ingredient that breaks time-reversal symmetry. Statistical mechanics does not have any such ingredient. To remove this symmetry, you need either time-asymmetric boundary conditions or time-asymmetric laws of physics (Callender 2011). In the absence of either of these ingredients, you have Loschmidt's paradox: for any system $\mathrm A$ that evolves from $t_1$ to $t_2$ so as to increase the entropy from $S_1$ to $S_2$, we can construct a system $\mathrm {A'}$ that starts with the particles in the positions they had at $t_2$, but with opposite momenta. The system will then evolve from $S_2$ to $S_1$.

What you can derive solely from statistical mechanics is a form of the second law that says that if a system experiences a large fluctuation away from equilibrium, then at sufficiently large times both before and after, it will, with high probability, be closer to equilibrium (Callender 2011). This is really just a statement of ergodicity, i.e., that all states are equally probable.

The standard interpretation of the second law today is that it arises from asymmetric boundary conditions. For reasons unknown to us, we had a low-entropy Big Bang.

Here is another question that this one very nearly duplicates. I wrote an answer there that spells out some of the ideas in more detail, for a specific toy system.

References

Callender, Craig, "Thermodynamic Asymmetry in Time", The Stanford Encyclopedia of Philosophy (Fall 2011 Edition), Edward N. Zalta (ed.), http://plato.stanford.edu/archives/fall2011/entries/time-thermo

• A low entropy at the Big Bang origin, does not explain, alone, the continuous increasing of entropy. It could have also decreased until a even lower entropy, for instance, then re-increase. – Trimok Oct 20 '13 at 15:49
• @Trimok: I stated a weak form of the second law in my second paragraph. This is discussed at greater length in the opening of section 2 of the Callender paper. Given the weak form of the second law, plus the low-entropy initial condition, the strong form of the second law follows. The breaking of the symmetry by the Big Bang is discussed in section 2.3. – Ben Crowell Oct 20 '13 at 17:56
• @Trimok: I added a link to a different answer in which I spelled out some of the stuff you're asking about. – Ben Crowell Oct 20 '13 at 19:52
• @BenCrowell I'm not entirely convinced by this. In particular, although I have never encountered a convincing argument, I have hope that someone out there can (perhaps has) or will demonstrate in the context of quantum statistical mechanics that the second law emerges as a consequence of unitary evolution coupled to a suitable notion/interpretation of entropy (i.e. von-Neumann entropy) and some argument about entropy increasing because of increase in correlations between quantum subsystems. I understand the time asymmetry argument, but no-go statements in physics are notoriously unreliable. – joshphysics Oct 21 '13 at 1:54
• @joshphysics I agree wholeheartedly about impossibility theorems in physics. I am highly intrigued by them in mathematics, so I'm drawn to them in physics, but there think of them more as "experimental evidence". For me in thermodynamics, experiment is queen: I've fried my brain enough over the years looking at "proofs" and such like. I don't believe we have a rigorous enough foundation of probability to "prove" much (aside from measure properties and so forth, which are only models). I smile when people ask "why can QM use classical probalities?" - like, do you reckon that'd be any easier! – WetSavannaAnimal Oct 22 '13 at 8:09

A relevant theorem here that seems (further to and separate from the Loschmidt paradox arguments spoken of by Ben Crowell) to weigh against a proof of the second law is the Poincaré Recurrence Theorem which, roughly speaking, a system (with certain assumptions) will, given enough time, evolve back to something arbitrarily near its beginning state. More precisely, quoting from the statement in Wikipedia.

Let $(X,\Sigma,\mu)$ be a finite measure space and let $f\colon X\to X$ be a measure-preserving transformation. ...

Theorem:

For any $E\in \Sigma$, the set of those points $x$ of $E$ such that $f^n(x)\notin E$ for all $n>0$ has zero measure. That is, almost every point of $E$ returns to $E$. In fact, almost every point returns infinitely often; "i.e."

$$\mu\left(\{x\in E:\mbox{ there exists } N \mbox{ such that } f^n(x)\notin E \mbox{ for all } n>N\}\right)=0.$$

or, informally, the measure of the set of points in phase space that are not at some time mapped back to themselves by the system's evolution have measure nought, or "there are almost no points which are not mapped back to themselves by some evolution of the system over time".

So how do we apply this to the Universe? We need some assumptions.

1. The Universe's phase space $X$ is a meaningful concept and it can be construed as a finite measure space, i.e (i) we can define a $\sigma$-algebra and a measure for it (ii) $X$ is the countable union of measurable sets with finite measure;
2. The measure in 1. is conserved by the laws of physics. This is wontedly taken to be true by people who believe in this argument for the Universe, because they construe the measure in 1. to be the phase volume measure and then Liouville's theorem (see Wiki page with this name) ensures it is conserved. Therefore we need to assume Liouville's theorem.

So, roughly speaking, a finite upper bound can be found on the "accessible" phase volume. If the Universe turns out to be finite spatially, then this would be reasonable, and that Liouville's theorem holds.

So, given certain reasonable sounding assumptions about the Universe, a proof for the second law of thermodynamics is a forlorn hope, because given enough time the Universe will come back to a state of any entropy it had in the past.

Of course, the assumptions show that there are several ways for this argument to fall, but a proof of the second law of thermodynamics would tell against at least one of the assumptions 1. and 2., so it would have interesting implications for other physics, allowed cosmological models and how their phase spaces work in particular.

• Oliver Penrose deals with the Poincaré recurrence theorem by observing that recurrence times depend on the initial coniditons and that the system might come back right away close to the origin or after the age of the universe. Equilibrium stat. mech. deals with that by allowing all possible microstates (even those that one would naively caracterize as "non equilibrium" like all particles in the corner of a box). – gatsu Oct 22 '13 at 12:08
• @gatsu Thanks for the link. I've not read Oliver Penrose, only his famouser brother, but it's on the to read list. Is his discussion in the lecture series on the web link you sent me? Yes I agree that you're likely talking about some fantastically long times for recurrence to happen, but if you're going to prove the second law consistent with the PRT, then it would have to be a law that says (i) entropy is almost certain to rise over timescale $T$ and (ii) give some estimate of $T$. Of course also as in my answer all bets are off if the Universe is pathologically infinite in breadth. – WetSavannaAnimal Oct 22 '13 at 12:57
• According to recent advances in out of equilibrium physics, what matters could be the average entropy variation and not the actual entropy for one given process. This was already the essence of Boltzmann's answers to Loschmidt and Zermelo's objections. My previous point was simply to say that what matters is that there exists an invariant probability distribution that maximizes an entropy functional and toward which almost all initial distributions will tend in the infinite time limit. – gatsu Oct 22 '13 at 13:20
• @gatsu I think I agree with you "what matters most ..." - is this the same as the "pointy distributions" in the section "A Random Walk Through Phase Space" of my answer here? That essentially there are maximum entropy states and not much else, so the system will always tend to be in these states most often just by dint of the "random walk". This of course also hinges on the ergodic hypothesis, which will not strictly be true in all cases owing to correlations between microstates, but, as I talk about in my answer, there are intuitive ... – WetSavannaAnimal Oct 22 '13 at 21:05
• ..arguments that it still holds at an appropriate level of coarse graining - so the correlations will not matter. Now all this is very old stuff (50s and 60s), essentially the work of E. T. Jaynes (whose papers I learnt most of my thermodynamics from), so it is way before the Searle's (or Bernhardt's) and company theorems, which I have glossed but not yet gone through deeply. Anyhow, I agree the "average" distribution must be the maxent one, but does that not just say Poincaré recurrence will only take the system away from this state for fleetingly short fractions of long term time? – WetSavannaAnimal Oct 22 '13 at 21:10

If you consider the postulate of the equal a priori probability, you will give the conclusion at the same time- tend to the most probable distribution. In statistical mechanics, tending to the most probable distribution is a probability, and for Boltzmann' entropy, $dS\ge 0$ is also a probability but not an inevitable result. So you can’t prove $dS\ge 0$ as an inevitable result from statistical mechanics.

On the other hand, the postulate of the equal a priori probability does not need to be considered for thermodynamics, please consider local non-equilibrium thermodynamics, in the equation

\begin{align}\frac{d_iS}{dt}=\sum_iJ_i·X_i\ge 0，\end{align}

some of the driving forces $X_i$ of the irreversible processes are not originated from the condition of the equal a priori probability, such as the gradient in generalized force $X_i =\nabla Y$, the gradient in chemical potential $X_i =\nabla\mu_j$, so the proof for the second law of thermodynamics from statistical mechanics will be incomplete.

This question is irrelevant to T-symmetry of physics. T-symmetric laws and T-asymmetric laws are the two different laws, the two describe different principles of physics. The key-point is that the theoretical structures of thermodynamics, statistical mechanics and dynamics are different. As the well known fact, the first law of thermodynamics is also a T-symmetric law.

\begin{align}dU=\delta Q+\delta W+ \sum_j\mu_jdN_j \end{align}

To doubt the second law of thermodynamics by T-symmetry of the first laws makes no sense, due to the two involve different principles of physics, and similarly, we also can not doubt the second law of thermodynamics by T-symmetric laws of dynamics. Time-symmetric laws of dynamics should compare with the first law of thermodynamics but not the second law.

How do you prove the second law of thermodynamics from statistical mechanics? and Mathematical proof of the Second Law of Thermodynamics are the two different questions!

The Second Law of Thermodynamics is an approximation, it has statistical or probabilistic validity. Statistical Mechanics corrects the plain flat out version of it that says entropy never decreases to the following.

The overwhelming majority of the time, a sufficiently large system which is not a closed system (in the sense of mechanics: note that in Thermodynamics, the phrase "closed system" has a different meaning than it does in Hamiltonian mechanics) but is in thermal contact with its environment, will, if it passes from one equilibrium state to another, not decrease its entropy.

Now, entropy in the sense of thermodynamics isn't even defined for states that are not equilibrium states. If the Universe as a whole is not in a state of equilibrium it does not possess a well-defined entropy. That goes double for its initial condition. Last time I looked at the Universe, it didn't look like it was in equilibrium. So the overwhelming empirical evidence for the truth of the Second Law in Thermodynamics says nothing about the Universe.

An excellent discussion of the issue between Zermelo (and Loschmidt) and Boltzmann about the H-theorem vs. reversibility and Poincaré recurrence is given in von Plato, Creating Modern Probability, and by Janneke van Lith (2001). Ergodic Theory, Interpretations of Probability and the Foundations of Statistical Mechanics, an excellent review paper. Because of the "probability" loopholes, Zermelos's objections to the naive interpretation of the Second Law do not apply to Boltzmann's eventually more nuanced statistical version.

Dr. van Lith's doctoral thesis is open access http://dspace.library.uu.nl/bitstream/handle/1874/657/full.pdf?sequence=1

as is her review of Guttmann's misch-masch http://www.projects.science.uu.nl/igg/dis/guttmann.html

First, let's introduce small subsystem of isolated system. The number $N$ particles of this subsystem is sufficiently large for interpretating subsystem as quasiclosed (fluctuations of macroscopic values are proportionally to $\frac{1}{\sqrt {N}}$). We may then say that function of distribution, according to Liouville theorem, is integral of motion of subsystem. So it is possible to say that function of distribution in this case is function of energy. So for energy distribution of subsystem it's possible to write $$dP_{E} = \int \limits_{E}^{E + dE} \rho (E)d\Gamma^{2n} = \rho(E)\int \limits_{E}^{E + dE}d\Gamma^{2n} =\rho (E)d\Gamma_{E} = \rho (E)\frac{d\Gamma_{E}}{dE}dE = \rho_{E}(E)dE, \qquad (.1)$$ where $d\Gamma^{2n}$ is the element of phase volume of subsystem ($2n$ refer to $6N$, where $N$),

$\rho (E)$ is a function of energy micro-distribution (i.e., possibility to find a subsystem in a state with energy $E, E + dE$ which corresponds to some impulse $p, p + dp$ from the element (!) of the phase volume), which is "almost" constant in the element $\Gamma^{2n}$, so we may carry it outside the integral,

$d\Gamma_{E} = \int \limits_{E}^{E + dE}d\Gamma^{2n}$ refer to the value of "spherical" layer which refers to impules $p, p + dp$ of subsystem.

$\rho_{E}(E)$ is the function of energy macro-distribution (i.e., to find subsystem in a state with energy $E, E + dE$ to which correspond all possible impulses $p, p + dp$ of phase volume).

Then, the energy of given quasiclosed system is almost constant and lies in a small neighborhood $\Delta E$ near average energy value $\langle E\rangle$ . This leads to the peak of distribution function for $E = \langle E\rangle$. It means that $$\rho_{E}(E)\Delta E \approx 1 .$$ By turning back to $(.1)$ it makes possible to write $$\rho (\langle E\rangle )\Delta \Gamma_{E} \approx 1,$$ where $\Delta \Gamma_{E}$ corresponds to element (not infinitesimally small) of phase volume in which the subsystem spend the most time. So it consist of information about full number of microscopic states of a subsystem, which create its macroscopic state with energy $\langle E\rangle$. Thus $\Delta \Gamma_{E}$ determines statistical weight $\Omega$ of subsystem: $$\Delta \Omega (\langle E\rangle) = a\Delta \Gamma_{E}.$$ Then $\Delta \Omega (\langle E\rangle)$ can be represented as product of $\omega_{i} (\langle E_{i}\rangle )$ of subsubsystems of the subsystem: $$\Delta \Omega (\langle E\rangle) = \omega_{1}(\langle E_{1}\rangle )...\omega_{m}(\langle E_{1}\rangle ).$$ So it's convenient to use logariphm of $\Delta \Omega (\langle E\rangle)$, which is called the entropy $S$: $$S = \ln(\Delta \Omega (\langle E\rangle)) = \ln(a\Delta \Gamma_{E}).$$ Let's back to $(.1)$. Function $\rho (E)$ changes slowly when compairing it with $\Delta\Gamma_{E}$. So for macroscopic states $$\Delta P_{E} \approx \Delta \Gamma_{E} = \frac{1}{a}e^{S}.$$ This leads to the next statement: the possibility for given macroscopic state of system increases when entropy increases. So for large subsystem possibility to move to a state with less entopy is strongly suppressed.

Then you only must to establish a link between entropy and heat and, finally, to get the second law of thermodynamics.

• Sorry, but -1. Except for the final three sentences, this answer only establishes that probability increases exponentially with increasing entropy. You then say, "This leads to the next statement: the possibility for given macroscopic state of system increases when entropy increases. So for large subsystem possibility to move to a state with less entopy is strongly suppressed." The second sentence doesn't follow from the first; it can't, because the first is time-reversal symmetric and the second isn't. See en.wikipedia.org/wiki/Loschmidt%27s_paradox . – Ben Crowell Oct 20 '13 at 15:24
• @BenCrowell . I don't understand why the second statement isn't the consequense of the first. Let's have non-equilbium closed system with energy $E$. We can share it out on small quasiclosed subsystems with energies $E_{i}, \quad \sum_{i}E_{i} = E$. Small subsystems reach an equilibrium (which may be escribed as the state with maximum entropy, which is a consequence of a definition of equilibrium) faster, than system. I.e., there is possible a case when there is an equilibrium in subsystems, but there isn't an equilibrium along the subsystems. So all system isn't in equilibrium. – user8817 Oct 20 '13 at 15:53
• @BenCrowell . But the possibility $dP(E)$ for system is the product of possibilities $dP_{i} \approx exp(S_{i}(\langle E_{i}\rangle))$ of small subsystems. If the system isn't at equilibrium, the energies $\langle E_{i}\rangle$ of it's subsystems may be interpreted as variable energies, by which the equilibrium is established. The greatest probability is reached when $S = S_{max}$. – user8817 Oct 20 '13 at 15:56
• @BenCrowell . So, if in initially state (which corresponds to initially moment of time) the closed system wasn't in equilibrium (it must be non-closed before initial moment, of course), it is the most possible that it goes to the state with monotonic increase of entropy, i.e., to the equilibrium state. It leads to the statement $\frac{dS}{dt} \geqslant 0$. – user8817 Oct 20 '13 at 16:00
• @BenCrowell: I agree with you, of course. However, it is still possible to derive irreversible behavior from reversible dynamics in suitable limits, provided one excludes sets of initial conditions of zero measure (which does break time-reversal symmetry). This is the case of all rigorous derivations of Boltzmann's equation (see, e.g., Lanford's famous work: Time evolution of large classical systems, or its papers in Comm. Math. Phys. 9 1968 176–191 and Comm. Math. Phys. 11 1968/1969 257–292). – Yvan Velenik Oct 21 '13 at 8:56

I think a satisfying reply to your answear would be rather extensive. I just give you a couple of references.

Regarding the second law of thermodynamics, it can be derived from statistical physics from the assumption of equal prior probability postulate, about which I suggest you to read chapter 1 of Reifs's book: Statistical mechanics

For what concerns the fact that Entropy can only increase with time I suggest you to have a look at the so-called "H-theorem" or "Irreversibility theorem", which you can find explained step-by-step here.

• Trimok's answer gives a sketch of the H-theorem. The H-theorem isn't really a proof of the second law, for the reasons given in my comment on Trimok's answer. The (anonymous?) author of the OCW notes you linked to gives a muddled hand-waving interpretation on pp. 4-5 and, IMO, completely fails to get at the underlying issue. – Ben Crowell Oct 20 '13 at 15:20

An idea of the explanation, while not completely rigourous.

Under the hypothesis of molecular chaos, one may consider two-body collisions $AB \leftrightarrow A'B'$, and we have, for this particular collision :

$$\frac{dp_{A'}}{dt} = \frac{dp_{B'}}{dt} = -\frac{dp_{A}}{dt} = - \frac{dp_{B}}{dt} = C_{A,B,A',B'} (p_{A}p_{B} - p_{A'}p_{B'})$$

where $C_{A,B,A',B'}$ is a positive constant.

Starting with $\sum p_I=1$, the variation of the entropy $S$ is $\frac{dS}{dt} = \sum -\log p_I \frac{dp_{I}}{dt}$

So, we get :

$$\frac{dS}{dt} = - \frac{1}{4}[\sum_{A'}(\log p_{A'}\frac{dp_{A'}}{dt})+\sum_{B'}(\log p_{B'}\frac{dp_{B'}}{dt}) +\sum_{A}(\log p_{A}\frac{dp_{A}}{dt}) + \sum_{B} (\log p_{B}\frac{dp_{B}}{dt})]$$

That is :

$$\frac{dS}{dt} = - \frac{1}{4}\sum_{A, B,A', B'}C_{A,B,A',B'}(\log p_{A'}+\log p_{B'} -\log p_{A}- \log p_{B}) (p_{A}p_{B} - p_{A'}p_{B'})$$

Finally :

$$\frac{dS}{dt} = \frac{1}{4}\sum_{A, B,A', B'}C_{A,B,A',B'}(\log p_{A'}p_{B'} -\log p_{A} p_{B}) (p_{A'}p_{B'} - p_{A}p_{B})$$

Because the $C$ are positive constants, and $log$ is a monotone function ($x > y \to \log x > \log y$), the above expression is positive :

$$\frac{dS}{dt} \geq 0$$