Qualitative discussion about entropy and disorder

Question

Many discussions about entropy and disorder use examples of decks of cards, pages of books thrown in the air, two gases being mixed in a container, even the state of a nursery at the end of the day compared to the beginning of the day in order to explain the idea of order (and disorder). In all these examples it is pointed out that the disorder of the system increases, or that the system is in an ordered state and finishes in a disordered state after something has happened. Take the case of throwing the pages of the book in the air. You start with the pages numbered in sequence (I didn't want to use the for "order"), you throw them in the air, they land on the floor, you collect them up and notice the pages are not in sequence anymore. ANd the point is "They are not in the sequence I call ordered. Nonetheless, they are in a new sequence." And, it appears to me that the probability to find the pages in this precise new sequence is equal to find them in the original sequence". In that sense, 'order' seems to be something that us humans define and it doesn't appear to be a property of the system. On the other hand, I can see that in the case of two gases mixing, empirically we find more states where the two types of molecules are occupying the entire volume of a container than one type of molecule in the left side and the other type in the right side of the container. Nonetheless, the precise state of each molecule, its position and therefore the entire state of the mixed up system is qualitative the same, isn't it? Isn't it equally difficult to make each molecule occupy that precise position in the mixed up state as in the unmixed state? Does this make sense?

Answer 1

I think a key observation here is that entropy is used when you are trying to describe a system on a macroscopic scale, which means you want to make predictions about macroscopic quantities. Using your example of two kinds of gas in a box (call them red and blue), before we talk about entropy of various states, we should consider what kinds of quantities are meaningful at a macroscopic scale. One set of quantities that are not macroscopically relevant are the exact positions of every molecule in the box. That information is not accessible when the system is "coarse-grained," or viewed macroscopically.

To see what information is accessible, lets assume that your coarse-graining only allows you to divide the box into two cells, a left half and a right half. You are interested in knowing if the gas tends to be "mixed" or "separated." The individual positions of the molecules are not observable, but what is observable is the number of molecules of each type on each side of the box. We could define a quantity that measures the mixedness $M$ of the particles as something like $M=(N_B-N_R)/N_\text{tot}$, where $N_{B/R}$ is the number of blue/red molecules in the cell, and $N_\text{tot}$ is the total number of particles in the cell.

Now we can ask which state is more probable: the one with blue on one side and red on the other (unmixed), or the one with nearly equal numbers of red and blue molecules (mixed). There is only one unmixed state (or I guess two since you could have all the red on the right or all the red on the left), but many many mixed states--remember you don't get to measure positions of individual particles, but just how many of each type are in each cell.

So it boils down to identifying what states are meaningful on the macroscopic scale, and then counting how many different ways there are to produce the macroscopic state. Then you can assign an entropy to the state which is bigger for states that are more probable (i.e. can be made in many different ways).

The example of pages in a book being mixed up is a bit more difficult to see how entropy could enter, but the key point is to identify the macroscopic quantities which characterize the state. For this, we shouldn't ask about the precise position of page 1, page 2, and so on. Instead, a good macroscopic quantity for measuring disorder could be the number of pages which ended up in the correct location. Then we see that there is only one possibility where all the pages are in the correct position, but many more possibilities where none of them are in the correct position, and hence the latter option is a state of higher entropy. You could be more sophisticated about how you define a disordered state (maybe look at how many even numbers are next to another even number and odd next to odd), but the key is to focus on macroscopic properties of the system, and not on the individual position of each page.

Answer 2

The basic assumption of statistical physics is, as you correctly assume, that all states are equally probable (there may be external reasons why some are less probable than others, but in the simplest case this holds). The concept of order and disorder, then, arises only when we "squint" and look at the system at a more abstract, macroscopic, level.

Example: Imagine that you divide the box of gas into two (imaginary) containers. If you have $N$ particles in total, and $k$ of them are to the left (hence $N-k$ to the right), there are $N \choose k$ or $\frac{N!}{k!(N-k)!}$ ways of defining such a state. For even moderately large numbers of particles, this is a huge number. Compare this to the case where you have all particles on one side; there is exactly one state describing this situation.

As for the example with the pages, you are correct in you assessment that the concept of order is not unique, but it is well defined: there are $N!$ ways of ordering $N$ pages, but only one where they are in a predefined order, and the deviations from this order can be assigned different probabilities.

A high entropy state is then a state which does not surprise us when we find it, whereas a state (such as all molecules being in one corner, or the pages being in a steadily growing order, when there are vastly more states where this is not true) is one of low probability, and thus surprising.