We've heard that entropy is defined for macrostates (fixed temperature, pressure, volume, etc...) in terms of the number of microstates that lead to said macrostate.

My question is: if we define macrostate and microstate more loosely, does all the thermodynamics behind entropy still hold? Does the Second Law still apply?

For example: If we consider a pair of coins and define a "macrostate" to be the number of heads and a "microstate" to be the state of both coins, will this lead to a valid definition of entropy of the system? S = k log(W) doesn't seem reasonable here. Will the Second Law hold? The coins aren't constantly jumbling around, so is it implicit in the Second Law that things must be changing state?

Are there any macroscopic systems where entropy can be defined in some way that's consistent with microscopic systems? Maybe the pair of coins bouncing around in a washing machine? Or is there some special property of atomic systems that is needed to define entropy?

  • $\begingroup$ The statistical definition of entropy (in terms of microstates) is consistent with the thermodynamic one when one takes the number of particles to be very very very large (for all practical purposes, infinity). Instead of two coins, if you take $10^{23}$ coins, the definition manifests itself as the macroscopic one. $\endgroup$ Commented Feb 7, 2020 at 22:27
  • $\begingroup$ @FellowTraveller But the coins aren't constantly changing state, meaning the Second Law shouldn't hold, right? $\endgroup$
    – dljs
    Commented Feb 7, 2020 at 22:31
  • $\begingroup$ You only know about the macrostate like say 70% of them are heads. But you do not know the configuration of each of the coins. So there can be many microstates leading to the 70% heads state. $\endgroup$ Commented Feb 7, 2020 at 22:37
  • $\begingroup$ @FellowTraveller If the macrostate is 70% heads, the entropy of that state is determined by the number microstates (like you said). But from here the entropy cannot change because the macrostate doesn't change. Also, the coins aren't switching between the different microstates that lead to the 70% heads macrostate, they are fixed in one microstate. So this would mean the coins aren't analogous to entropy applied to microscopic things, right? $\endgroup$
    – dljs
    Commented Feb 8, 2020 at 1:52
  • $\begingroup$ Microstates aren’t fixed. They could be switching between all the ones that give you the right macrostate. And if a macrostate is fixed that means the system is in equilibrium. So entropy shouldn’t change. Entropy only changes if the system isn’t in equilibrium. $\endgroup$ Commented Feb 8, 2020 at 4:46

1 Answer 1


Establishing a relation between different "entropies" is less trivial than it appears from the choice of the same naming. Unfortunately, even in scientific papers and textbooks, it is quit easy to find careless and unjustified identifications without specifying their limits.

Information theory provides a very general expression for the average information to be assigned to a probability distribution $\{p_i\}$, the Shannon's formula: $$ H_s = -\sum_i p_i \mathrm{log}p_i. $$ Due to its similarity with the Gibbs-Boltzmann statistical mechanics expression, this average information was also named Information Entropy. One of the reasons is that it reduces to the Gibbs-Boltzmann entropy (a part a trivial constant factor depending on the choice of units) if the probability distribution coincides with any of the statistical ensemble distributions of Statistical Mechanics. Therefore, at the thermodynamic limit, one can get contact with thermodynamic entropy.

A key step in order to establish link to equilibrium thermodynamics is then played by the probability distributions of the statistical ensembles describing system at thermodynamic equilibrium. It is a consequence of Liouville's theorem that any equilibrium probability density of Hamiltonian systems must depend on the microscopic dynamical variables trough the Hamiltonian and that is precisely where the special link between thermodynamic entropy and energy is established (do not forget that thermodynamic entropy differences are routinely measured with calorimeters!)

Shannon's formula is actually much more general, since it does not require a Hamiltonian system, and in principle not even a stationary probability distribution. So, it could be applied to encode into an information entropy the information of a probability distribution of outcomes, provided a macroscopic state has been selected by a wise choice of an macroscopic observable like the total number of heads.

However, there is no natural way to make contact with the usual thermodynamics. First of all, there is no natural way to speak about equilibrium. The main reason being the lack of an underlying dynamics. Let's notice that the existence of such a dynamics would be a necessary but not sufficient condition for the existence of a thermodynamic-like entropy. A second, equally important, condition would be that the macroscopic observable(s) used to characterize the statistical ensemble should be preserved by this dynamics. The proposed example of coins bouncing around in a washing machine does introduce a dynamics, but does not preserve the observable "number of heads". Nevertheless, it could be suitably modified. For example, by adding an automatic control which counts a new microscopic configuration only when there is a given number of heads.

Notice that even in such a case, missing a physically meaningful energetic description of the macrostate, there is no way to make full contact with usual thermodynamics without additional hypotheses or further modification of the model.

  • $\begingroup$ "the macroscopic observable(s) used to characterize the statistical ensemble should be preserved by this dynamics". I would only agree that this is the case at equilibrium. For example in the thermodynamic limit (lots of coins in the washer), initially the macrostate may be 70% heads but as the washer tumbles the macrostate won't be preserved and will move towards equilibrium (50% heads). In this case (again for a ton of coins), the system will be constantly flicking among all the microstates while staying about 50% heads. What other conditions must be met to make contact with thermo? $\endgroup$
    – dljs
    Commented Feb 9, 2020 at 18:42
  • $\begingroup$ @dljs Thermodynamic entropy is clearly defined and characterized for equilibrium systems with an energy. Without energy, only some analogy is allowed. $\endgroup$ Commented Feb 9, 2020 at 21:17
  • $\begingroup$ Okay, so far I'm getting that we need 1. Dynamics for constantly switching between microstates. (does it matter how it switches? Can it be orderly switching or must it be random? Does the frequency of switching matter?) 2. An Energy. (must this be the actual energy, or can we define "energy" for the system in some other way like we did for "macrostate" and "microstate"?) $\endgroup$
    – dljs
    Commented Feb 10, 2020 at 12:59
  • 1
    $\begingroup$ @dljs 1. ok. I would say that the kind of dynamics is largely arbitrary, although some ergodic-like property is necessary to justify equivalence of an ensemble description with time averages. 2. If you want to allow thermal contact with usual lab systems, energy must be the energy of the system. $\endgroup$ Commented Feb 10, 2020 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.