The definition of entropy, be it Boltzmanns ($k_B \text{ln } \Omega $), Gibbs ($ - k_B \sum_i p_i \text{ln } p_i $) or Von-Neumann's ($ - k_B \text{Tr} \hat{\rho} \text{ln } \hat{\rho} $) always relies at least on some probability interpretation, (in the first case, it actually only applies to a system in equilibrium states). In the quantum mechanical case, there isn't an ensemble, but the classical probabilities manifest in the density operator.

Given these formulae, my first impression is that we can only talk about the quantity 'entropy', if we model a system as an ensemble and talk about probabilities.

I'd like to know if there is a definition of entropy that also applies to a micro-state. If that's not the case, what do we actually mean by the statement that 'the entropy of an isolated system increases with time'? What does it mean, especially if we (because we are some kind of omniscient entity) presume to know position and momentum of every particles?

Before I could say something about the entropy of this system, I would have to find an appropriate probability distribution for an ensemble describing the system. My first guess for such a probability distribution would be the one which models the timely behavior of the system. By 'timely behaviour' I mean that the probability density which I choose correctly recreates all the average values that I would also get by measuring the corresponding observables at a selection of different instants in time.

Would that be the right way to go?


1 Answer 1


This gets exactly into what is called "coarse graining." The idea goes as follows. Say you have some computer simulation where you know the position and momentum of every particle in a large box. There is indeed no way to assign an entropy to this microstate. If you know the microstate, then $\Omega = 1$, and $S = k \ln(\Omega) = k \ln(1) = 0$.

However, its sort of ridiculous to say you literally know all this data. What is maybe more reasonable is the following. First, break the large box up into a grid of a million tiny cubes of volume $1 \mathrm{mm}^3$. Maybe you know the energy $u$ and number of particles $n$ in each of these million tiny $1 \mathrm{mm}^3$ cubes. Then, in your simulation, say you calculate $u$ and $n$ for each tiny cube, i.e. you figure out the macrostate corresponding to the tiny cube's microstate.

You then calculate the entropy of that tiny cube's macrostate, using say the Sackur-Tetrode formula. You then sum up all the entropies of all the tiny cubes to get the overall entropy of the large box. This procedure is what is called 'coarse graining,' where the tiny yet finitely sized boxes are perhaps the 'coarse grains.'

What you'll see in your simulation is that the total entropy of the box will indeed increase in time. (This is just because the whole box's true microstate, which is essentially wandering the phase space randomly, is more likely to wander into a macrostate with more phase space volume.) You could start the simulation with all the particles in one corner of the box, and overtime they will go to occupy the whole thing homogenously. The distribution of number density/energy density/entropy density over the tiny boxes will smooth out in time, i.e. the gradients will dissipate.

Coarse graining perhaps gives us a mentally clear picture of what we mean when we say that entropy always increases in regular everyday objects.

  • $\begingroup$ So you model the "microstate" with a bunch of appropriately choosen macrostates in equilibrium (and because the probability distribution of those is known, this is how one achieves the probabilities needed to talk about entropy? $\endgroup$ Commented Apr 13, 2021 at 14:45
  • $\begingroup$ Yes. Although I don't think its actually correct to say the tiny boxes are in equilibrium, because what are they in equilibrium with? Its just that you are labelling the tiny box's microstate with two numbers, $u$ and $n$, and the macrostate consists of all microstates with those values of energy and particle number. You also might want to look up Boltzmann's H theorem. Using the "molecular chaos hypothesis," one is able to prove that this coarse grained entropy of the whole gas increases with time. $\endgroup$ Commented Apr 13, 2021 at 16:15
  • $\begingroup$ I meant that just for themselves, the tiny boxes are in an equilibrium state (if they weren't, we wouldn't know how to relate n and u to the entropy of every tiny box) $\endgroup$ Commented Apr 13, 2021 at 18:17
  • $\begingroup$ I would say you should ask yourself carefully what it means for a tiny box to be in equilibrium. A macrostate is just a finite volume region of phase space microstates. A macrostate labelled by $u$ and $n$ just contains all microstates of energy $u$ and number $n$ with equal probability. What would allow you to say that one such microstate is any more in equilibrium than another? The word "equilibrium" applies to two or more different systems who have maximized their combined entropy. I would say the little boxes could be in equilibrium with their neighbors but not themselves. $\endgroup$ Commented Apr 13, 2021 at 18:43
  • $\begingroup$ The word "statistical equilibrium" is as well used to describe a property of a macro state - in case of a microcanonical ensemble it is exactly the property that each accessible micro-state is equally probable. That's what I meant $\endgroup$ Commented Apr 13, 2021 at 19:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.