Loosely speaking,

  • The second law of thermodynamics (SLT) says that as a closed system evolves, its macrostate tends toward the one with the greatest possible number of indistinguishable microstates.

  • The law of large numbers (LLN) says that as the number of observations of a random event increases, the average of the observed values tends toward their expected value... which is precisely $1/n$ of the value that has the greatest possible number of ways in which it could be obtained through summing $n$ observations.1

The way I see it, if you realize that "evolution" (passage of time, a physical concept) is really just "having more opportunities for observations" (observing more samples, a mathematical concept), then these statements are saying the same thing.

Or, to put it another way, the SLT is implied by the LLN. Or to be completely blunt:
It is mathematically impossible for the 2nd law of thermodynamics not to hold.

Based on this, my questions are:

  1. Am I correct in my reasoning/intuition here?

  2. If yes: Why is the 2nd law of thermodynamics even considered a physical law at all? Provable statements aren't physical laws (nobody calls the Pythagorean theorem a physical law), so what sense does it make to regard this as one?

    If no: Why not?


I just remembered there's also the data-processing inequality (DPI), which states that, if $X \to Y \to Z$ is a Markov chain (formally meaning that $X$ and $Z$ are conditionally independent given $Y$, and informally meaning that "$X$ only influences $Z$ via an intermediate $Y$"), then $H(Z \vert Y) \leq H(Z \vert X)$, meaning that the additional entropy (information content) of knowing $Z$ is lower if we already know $Y$ rather than if we only know $X$. I feel like this might help with the above interpretation but it's not entirely obvious to me exactly how.

1 For what it's worth, the asymptotic equipartition property (AEP) in information theory is the analog of the LLN in that field, and given that it directly deals with entropy just like the SLT, it might make more sense for those familiar with it to use that as a comparison point, but I'll focus on the LLN here since it's more accessible (including to myself!).

  • $\begingroup$ I was told that indeed thermodynamic is not a theory although I am always confused on these things. I think there is a fine difference between postulates and observations made into laws. But it might be due to how a theory/field developed historically. $\endgroup$
    – Alchimista
    Commented Apr 10, 2018 at 8:58
  • 1
    $\begingroup$ The claim you labeled as "second law of thermodynamics" is, as far as we know, true, but it is not the Second law of thermodynamics. Second law says what is impossible to achieve, however violent the states the system may be put into, not what does happen naturally in the course of time. For the actual second law, see the Kelvin, Clausius, Planck statements on the Wikipedia page. Yours statement is describing the empiric fact that usually, systems left to themselves and isolated from other bodies tend to achieve thermodynamic equilibrium. This observation/assumption is different from 2nd law. $\endgroup$ Commented Apr 10, 2018 at 11:37
  • $\begingroup$ I do not entirely understand your argument, but there are big problems with assumptions. If a system evolves towards equilibrium but no observations are made, does LLN have anything to say about this evolution? $\endgroup$ Commented Apr 10, 2018 at 11:42

2 Answers 2


Personally, I agree with you that SLT and LLN do observe similarities. Nevertheless, there are still differences in between.

As you mentioned, LLN is a theorem. Being a theorem in mathematics, it requires rigorous conditions. Should these conditions be violated, the theorem would fail. Here are two crucial points that LLN concerns.

(1) For either the strong or the weak version, LLN focuses on an infinite sequence of random variables $X_1$, $X_2$, ..., $X_n$, ... Their sub-indices seem to play the role of time, i.e., $\left\{X_m\right\}_{m=1}^{\infty}$ can be regarded as a stochastic process. Yet this process takes discrete time. By contrast, the evolution that SLT concerns is more about continuous time than the discrete one. (Maybe there could be a parallel LLN for, say, martingales with continuous time; to the best of my knowledge, we seem not to have had such a theorem so far.)

(2) LLN requires that the random variables $X_1$, $X_2$, ..., $X_n$, ... are independent and identically distributed (i.i.d.). If you regard $X_i$ as the observation of a closed system you take at time $i$, this i.i.d. requirement goes that this closed system must be in its steady or stationary state (i.e., the system could be dynamic, but the dynamics itself should be time-independent, just like a steady flow of fluids). By contrast, SLT considers a much wider range of possibilities; it is believed to hold even for evolutionary closed systems.

Get back to SLT. It is fair to say that it grows on some provable facts (like the LLN analogy), and generalizes to the non-provable cases. Mathematically, it could be stated as that any closed evolutionary system has an associated real-valued state function $S$ that yields ${\rm d}S\ge 0$, where $S$ here is the well-known entropy. You could see that the way this statement goes very much resembles the Principle of Relativity. Perhaps this is why SLT should be considered as a law rather than a theorem. Its non-provability lies in that, while we could prove the existence of $S$ for some special ones, it would be impossible to go over all closed evolutionary systems.

  • $\begingroup$ LLN does not require i.i.d.; that's just the simplest version of it. See here for one generalization. See here and here for another. $\endgroup$
    – user541686
    Commented Apr 10, 2018 at 8:37
  • $\begingroup$ Well, there is a major additional distinction: the only randomness in the SLT is in the initial condition, the subsequent evolution being deterministic. This, of course, immediately disconnects it from the settings in which the LLN applies. $\endgroup$ Commented Apr 10, 2018 at 9:16
  • $\begingroup$ @YvanVelenik: I think you're making the same mistake? LLN does not require i.i.d. The deterministic case would just be a sequence of random variables that aren't actually random (i.e. they can only ever take on one value), which means they have bounded variance (zero), which I believe is sufficient for LLN to apply. $\endgroup$
    – user541686
    Commented Apr 10, 2018 at 9:43
  • $\begingroup$ @Mehrdad: I never assumed iid. What is called the LLN, however, always assumes randomness. Deriving analogues of the LLN (for example, some forms of the ergodic theorem) for deterministic systems can be done in some situations (with strong enough assumptions), but this is not what is called the LLN. $\endgroup$ Commented Apr 10, 2018 at 9:50
  • 1
    $\begingroup$ @Mehrdad : Moreover, it is essential to say what you mean by LLN if you want to claim (as in your post) that it implies the SLT (which it does not, for any existing version of the LLN). $\endgroup$ Commented Apr 10, 2018 at 9:58
  1. Your reasoning/intuition is partially correct...

    If yes: Why is the 2nd law of thermodynamics even considered a physical law at all? Provable statements aren't physical laws (nobody calls the Pythagorean theorem a physical law), so what sense does it make to regard this as one?

The Pythagorean theorem you mention is only a mathematical theorem if the 'triangles' and 'lengths' it refers to are defined within a particular abstract system of axioms. If you make the assertion that the same relation holds for triangles drawn on paper and measured by a ruler then what you have is a physical law (and one that is only approximately correct).

Similarly, the second law of thermodynamics, as a physical law, is something that can only be tested through experiment. It may be possible to find a particular consistent mathematical model of the physical world in which the the second law (as it is defined within the model) can be removed and then derived mathematically from the remaining definitions/assumptions of the model. Then you could argue that the second law is redundant within that model in a mathematically precise sense. However, the second law of thermodynamics would remain a perfectly good physical law.

If no: Why not?

The problem is it is not clear that the mathematical models we have in which the second law can be 'derived' are sufficiently general enough to well-describe all possible physical systems. People have worked on this of course, for example it was the main motivation for the development of ergodic theory: within the framework of Hamiltonian mechanics (say) if you can show that your Hamiltonian has this 'ergodic' property, then it follows something like the second law holds for the system. Problem: there are many physical systems which are not well-described by ergodic Hamiltonian dynamics.

This is where the second issue with your argument comes in. In order to invoke the LLN you claim that the physical system should be well described by a finite set of microstates, with repeated measurements of the system represented by a random sampling of these microstates. This works very well for some systems (indeed it is a common starting assumption in statistical mechanics), but it is not clear why it should be true for all systems.

As far as we can tell, the second law of thermodynamics holds for all physical systems. The hard part is showing that mathematical models in which we can 'derive' something like the second law cover all these possibilities.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.