# Ensembles in statistical mechanics

I've read the concept of an ensemble is like taking snapshots of the system under consideration in different intervals of time so you get a collection of all possible microstates of the system. Since entropy is directly proportional to the number of microstates a system can take on , what's the difference between entropy and ensembles?

• Entropy is $k \log \Omega$, not just proportial to $\Omega$.
– Dan
Commented Jan 1, 2022 at 13:57

An Ensemble is a group of systems that are microscopically different but macroscopically the same. For example, for an isolated box containing 1 mol of gas, which means we have around $$6\times10^{23}$$ molecules. The macroscopic state could be $$(E,V,N)$$ Energy, Volume, and particle number. The microscopic state could be the position and momentum of all $$6\times10^{23}$$ molecules, $$(\mathbf{r}_1,\mathbf{r}_2,...,\mathbf{r}_N,\mathbf{p}_1,\mathbf{p}_2,...,\mathbf{p}_N)$$.

The ensemble corresponding to our choice of macroscopic state, $$(E,V,N)$$, is called Microcanonical Ensemble. This include all microscopic states that are compatible with $$(E,V,N)$$. It's easy to be compatible with $$N$$ and $$V$$, just put in exactly $$N$$ number of particles and fix the volume of the box to $$V$$. To be compatible with $$E$$, first, your system can't change energy, so it must be isolated from other energy sources, i.e., we have to thermally insulate it. Second, all the microscopic quantities $$(\mathbf{r}_1,\mathbf{r}_2,...,\mathbf{r}_N,\mathbf{p}_1,\mathbf{p}_2,...,\mathbf{p}_N)$$ must be such that the energy they have as a whole $$H(\mathbf{r}_1,\mathbf{r}_2,...,\mathbf{r}_N,\mathbf{p}_1,\mathbf{p}_2,...,\mathbf{p}_N)=E$$ matches the energy $$E$$. In this sense, you can think of microcanonical ensemble as a energy contour on the space of all possible particle positions and momentum $$(\mathbf{r}_1,\mathbf{r}_2,...,\mathbf{r}_N,\mathbf{p}_1,\mathbf{p}_2,...,\mathbf{p}_N)$$, that is called the phase space.

Entropy measures the "size" of the ensemble with a logarithmic scale. You basically count how many set of position and momentum $$(\mathbf{r}_1,\mathbf{r}_2,...,\mathbf{r}_N,\mathbf{p}_1,\mathbf{p}_2,...,\mathbf{p}_N)$$ are compatible with $$(E,V,N)$$, take log scale and multiply by a constant.

A statistical ensemble is a set of systems which are so much alike that you either cannot distinguish them or do not care to. For an example of the first case, think of 4 electrons in a magnetic field, 3 of which have spin up and one has spin down. The electrons are indistinguishable, so you cannot say which has spin down, but since the magnetic moments caused by the spins have an energy in the magnetic field, as long as no energy leaves or enters the system, you can be sure that any measurement would yield 3 times spin up and 1 time spin down.

The second case can be illustrated by separating the electrons in space, thus making them distinguishable by their position. If you still only care for the total energy (the macrostate), you can continue to call the different possibilities of assigning spin up and spin down (the microstates) to the electrons an ensemble.

Entropy, on the other hand, is a measure of how much chaos there is, or, more precisely, how much information one is lacking to totally determine the state of a system. If you again consider 4 spacially separated electrons in a magnetic field with a given total energy $$E = 3 E_\uparrow + E_\downarrow$$, where $$E_{\uparrow(\downarrow)}$$ is the energy of an electron with spin up (down), the (information) entropy is $$\log_2(4) = 2$$, because you need 2 bit to represent any of the numbers 1,2,3,4, with which you can say which electron has spin down. The physical entropy would then be $$(k \log 4)$$ with $$k$$ as Boltzmann's constant, which is just a renormalisation compared to the information entropy. Physicists do this renormalisation, because we tend to be rather picky with respect to the units of quantities.

In general, when you have an ensemble of $$\Omega$$ microstates, you need $$\log_2 \Omega$$ bit of information to identify a microstate, so by just knowing the system is in any one of these microstates, you are lacking $$\log_2 \Omega$$ bits to fully determine its state. Renomalisation yields the physical entropy $$S = k \log \Omega$$.

The ensemble is a probability distribution, and the entropy is a measure of the uncertainty or diversity within this distribution.

Because microscopic observables (e.g., particle energy) in a bounded system are quantized, there is a discrete set of basis states compatible with given values of macroscopic observables (e.g., total energy). The "snapshots" you describe are drawn from a discrete probability distribution over these basis states.

The simple relation $$S = k \log \Omega$$ applies to the microcanonical ensemble (an isolated system), where there are finitely many such basis states ($$\Omega$$) and the distribution is uniform -- each state $$i$$ has probability $$p_i = 1/\Omega$$. This is an instance of the more general definition of entropy for discrete probability distributions, $$S = -k \sum_i p_i \log p_i.$$