Isn't the second law of thermodynamics just the law of large numbers in disguise?

Question

Loosely speaking,

• The second law of thermodynamics (SLT) says that as a closed system evolves, its macrostate tends toward the one with the greatest possible number of indistinguishable microstates.

• The law of large numbers (LLN) says that as the number of observations of a random event increases, the average of the observed values tends toward their expected value... which is precisely $1/n$ of the value that has the greatest possible number of ways in which it could be obtained through summing $n$ observations.¹

The way I see it, if you realize that "evolution" (passage of time, a physical concept) is really just "having more opportunities for observations" (observing more samples, a mathematical concept), then these statements are saying the same thing.

Or, to put it another way, the SLT is implied by the LLN. Or to be completely blunt:
It is mathematically impossible for the 2^nd law of thermodynamics not to hold.

Based on this, my questions are:

1. Am I correct in my reasoning/intuition here?

2. If yes: Why is the 2^nd law of thermodynamics even considered a physical law at all? Provable statements aren't physical laws (nobody calls the Pythagorean theorem a physical law), so what sense does it make to regard this as one?

   If no: Why not?

---

Edit:

I just remembered there's also the data-processing inequality (DPI), which states that, if $X \to Y \to Z$ is a Markov chain (formally meaning that $X$ and $Z$ are conditionally independent given $Y$, and informally meaning that "$X$ only influences $Z$ via an intermediate $Y$"), then $H(Z \vert Y) \leq H(Z \vert X)$, meaning that the additional entropy (information content) of knowing $Z$ is lower if we already know $Y$ rather than if we only know $X$. I feel like this might help with the above interpretation but it's not entirely obvious to me exactly how.

_{¹ For what it's worth, the asymptotic equipartition property (AEP) in information theory is the analog of the LLN in that field, and given that it directly deals with entropy just like the SLT, it might make more sense for those familiar with it to use that as a comparison point, but I'll focus on the LLN here since it's more accessible (including to myself!).}

Answer 1

Personally, I agree with you that SLT and LLN do observe similarities. Nevertheless, there are still differences in between.

As you mentioned, LLN is a theorem. Being a theorem in mathematics, it requires rigorous conditions. Should these conditions be violated, the theorem would fail. Here are two crucial points that LLN concerns.

(1) For either the strong or the weak version, LLN focuses on an infinite sequence of random variables $X_1$, $X_2$, ..., $X_n$, ... Their sub-indices seem to play the role of time, i.e., $\left\{X_m\right\}_{m=1}^{\infty}$ can be regarded as a stochastic process. Yet this process takes discrete time. By contrast, the evolution that SLT concerns is more about continuous time than the discrete one. (Maybe there could be a parallel LLN for, say, martingales with continuous time; to the best of my knowledge, we seem not to have had such a theorem so far.)

(2) LLN requires that the random variables $X_1$, $X_2$, ..., $X_n$, ... are independent and identically distributed (i.i.d.). If you regard $X_i$ as the observation of a closed system you take at time $i$, this i.i.d. requirement goes that this closed system must be in its steady or stationary state (i.e., the system could be dynamic, but the dynamics itself should be time-independent, just like a steady flow of fluids). By contrast, SLT considers a much wider range of possibilities; it is believed to hold even for evolutionary closed systems.

Get back to SLT. It is fair to say that it grows on some provable facts (like the LLN analogy), and generalizes to the non-provable cases. Mathematically, it could be stated as that any closed evolutionary system has an associated real-valued state function $S$ that yields ${\rm d}S\ge 0$, where $S$ here is the well-known entropy. You could see that the way this statement goes very much resembles the Principle of Relativity. Perhaps this is why SLT should be considered as a law rather than a theorem. Its non-provability lies in that, while we could prove the existence of $S$ for some special ones, it would be impossible to go over all closed evolutionary systems.

Answer 2

1. Your reasoning/intuition is partially correct...

   If yes: Why is the 2nd law of thermodynamics even considered a physical law at all? Provable statements aren't physical laws (nobody calls the Pythagorean theorem a physical law), so what sense does it make to regard this as one?

The Pythagorean theorem you mention is only a mathematical theorem if the 'triangles' and 'lengths' it refers to are defined within a particular abstract system of axioms. If you make the assertion that the same relation holds for triangles drawn on paper and measured by a ruler then what you have is a physical law (and one that is only approximately correct).

Similarly, the second law of thermodynamics, as a physical law, is something that can only be tested through experiment. It may be possible to find a particular consistent mathematical model of the physical world in which the the second law (as it is defined within the model) can be removed and then derived mathematically from the remaining definitions/assumptions of the model. Then you could argue that the second law is redundant within that model in a mathematically precise sense. However, the second law of thermodynamics would remain a perfectly good physical law.

If no: Why not?

The problem is it is not clear that the mathematical models we have in which the second law can be 'derived' are sufficiently general enough to well-describe all possible physical systems. People have worked on this of course, for example it was the main motivation for the development of ergodic theory: within the framework of Hamiltonian mechanics (say) if you can show that your Hamiltonian has this 'ergodic' property, then it follows something like the second law holds for the system. Problem: there are many physical systems which are not well-described by ergodic Hamiltonian dynamics.

This is where the second issue with your argument comes in. In order to invoke the LLN you claim that the physical system should be well described by a finite set of microstates, with repeated measurements of the system represented by a random sampling of these microstates. This works very well for some systems (indeed it is a common starting assumption in statistical mechanics), but it is not clear why it should be true for all systems.

As far as we can tell, the second law of thermodynamics holds for all physical systems. The hard part is showing that mathematical models in which we can 'derive' something like the second law cover all these possibilities.