How to count microstates in quantum statistical mechanics

According to the fundamental postulate of statistical mechanics (as far as I know it), we take the (classical) probability for a system to be in any of its microstates to be equal (if the system is in equilibrium and isolated). My question is, how do we count the number of microstates? From what I can see, the number of microstates is usually taken to be the number of linearly independant energy eigenstates of the Hamiltonian for that particular value of energy. However, I don't see why this is the 'number' of microstates. Are we supposed to imagine the microcanonical ensemble as a macroscopic quantum system for which we have measured the energy? In that case the system could be in any one of an (uncountably) infinite number of linear superpositions of its degenerate energy eigenstates. In other words the state vector could be be any vector at all in the degenerate energy eigenspace.

So why are we allowed to count the number of states as the number of linearly independant energy eigenstates, when we don't know that the system would even be in an eigenstate (but rather only in some linear combination)? And as a direct consequence of this method of counting, it seems we assign a non-zero probability only for the system to be in energy eigenstates, and ignore the possibility that it could be in a superposition.

• Welcome to Physics Stack Exchange. I just wanted to express admiration for a clearly written and interesting first post :) – DanielSank Sep 16 '16 at 13:59

Summary: the question "what is the probability of state $|\psi \rangle$" is the wrong question to ask, because it's experimentally unobservable. What you really care about is the results of measurements, and the number of possible different measurement outcomes is equal to the number of microstates.

Consider a bunch of noninteracting spin 1/2 particles in thermal equilibrium, and look at a single spin inside.

On the classical level, at thermal equilibrium, the spin of this particle should be a vector with random direction. Now we might ask, on the quantum level, what's the state of this particle?

However, this is the wrong question to ask. Given any spinor $|\psi \rangle$, it's possible to find an axis $\mathbf{n}$ so that $|\psi\rangle$ is the positive eigenstate of $S_{\mathbf{n}}$, i.e. the spin is definitely up along $\mathbf{n}$. So individual elements of the Hilbert space are insufficient to represent a thermal state.

Instead, we need to do the same thing we did in the classical case: replace the state of a system (i.e. $(\mathbf{x}, \mathbf{p})$ classically and $|\psi \rangle$ in quantum) with a probability distribution over those states. One such probability distribution is $$50\% \text{ chance } |\uparrow \rangle, \quad 50\% \text{ chance } |\downarrow \rangle.$$ The miraculous thing is that this distribution is rotationally invariant! That is, the distributions $$50\% \text{ chance } | \mathbf{n} \rangle, \quad 50\% \text{ chance } |-\mathbf{n}\rangle$$ are indistinguishable by measurement for any direction $\mathbf{n}$. Every spin measurement of any $\mathbf{n}$ ensemble, along any direction $\mathbf{m}$, gives a 50/50 result.

This tells us that the correct description of a thermal ensemble of quantum particles can't be an assignment of probabilities to quantum states, because such a construction isn't unique: there are many probability distributions that are experimentally indistinguishable. (The quantity that corresponds to probability distributions "up to distinguishability" is called the density matrix $\rho$.)

• Why don't we consider superpositions? We do! You can't directly ask "what is the probability the system is in state $|\psi \rangle$", because this is not an experimentally observable quantity, as explained above. But you can perform a measurement of an operator $\hat{A}$ with eigenvector $|\psi \rangle$ and eigenvalue $\lambda$, and there's some probability you get $\lambda$.
• Why do we count microstates? This is the dimension of our Hilbert space, or more physically, the number of different numbers you can get while measuring $\hat{A}$. In the example above, the probability of getting $\lambda$ is $1/2$ where $2$ is the number of microstates. We aren't privileging the spin up or spin down states, because you get a 50/50 chance for any spin operator $S_{\mathbf{n}}$.
• Why do we count specifically energy eigenstates? This is just for convenience. What we really want is the dimension of the Hilbert space, and $\hat{H}$ is often easy to deal with/reason about.
• This is a good answer, +1. The only thing I would add is some discussion of why the density matrix is the right thing to use for a thermal state. Since that's discussed elsewhere though, a link would be nice. – DanielSank Sep 16 '16 at 19:43
• number of different numbers you can get — but it isn't the dimension of our Hilbert space if there are degenerate states, since the latter is the more like number of "usually different" numbers you can get. – Ruslan Sep 16 '16 at 20:06
• @Ruslan Mhm, agreed. I skimmed over that to save a little bit of space; one can also read the phase as "the maximum number of different numbers you can get". – knzhou Sep 16 '16 at 20:15
• Thanks for the answer, +1. I am still having some trouble following the argument though. What exactly is the motivation for choosing the mixed state (and by mixed state I mean classical probability distribution over quantum states, correct me if I'm wrong) 50% up and 50% down? Is it simply that this state is rotationally invariant and hence doesn't 'privilege' any particular direction, and is in that sense fair? If that is the case I have trouble deciding how to choose my probability distribution in cases other than spin 1/2 particles (ideal quantum gas for example). – Andrew Sep 17 '16 at 4:32
• @NandagopalM Sure, that's a reasonable way to argue that the thermal density matrix ends up diagonal in the energy basis. But that doesn't stop you from considering the thermal density matrix in other bases, such as the position basis. – knzhou Apr 3 '19 at 8:47

First of all, independently of the ensamble, if you want a density matrix that describes a state of equilibrium, then it has to commute with the hamiltonian (see Liouville theorem for QM). This means that both the density matrix and the hamiltonian can be diagonalized in the same basis and, thus, the system is indeed in a statistichal mixture of energy eigenstates.

Now, if the system is in the microcanonical ensemble then "counting microstates" means counting how many non-zero terms does the density matrix has in its diagonal. That's why you don't take into account other possible states like superpositions of energy eigenstates. You need to count the diagonal terms of the density matrix written in a particular basis that diagonalizes the hamiltonian too.

• Please see my question physics.stackexchange.com/questions/293745/… for how counting superpositions of eigenstates (ie off diagonal elements) still results in stationary equilibrium state but does not give the correct QM answer. – ComptonScattering Nov 21 '16 at 12:27
• What a nice answer it is! After studying classical and quantum microcanonical ensemble I was solely confused by the counting procedure: where is my $\pho$ etc.! Now I see that connection. – rainman Feb 6 '18 at 23:54
• @P. C. Spaniel: Is the number of microstates = the number of diagonal terms in the density matrix? or, the number of nonzero diagonal elements in the density matrix? – rainman Feb 7 '18 at 0:55

The foundations of statistical mechanics are sometimes easier to understand for classical finite-state systems, such as a lattice of classical two-state systems (bits) used to model spins in a magnetic material. With $n$ bits you have $2^n$ distinct possible states. If there are macroscopic constraints such as external fields biasing the probabilities for each bit to be a 0 or 1, the effective number of bits is smaller.

Quantum Statistical Mechanics can be thought of as generalizing this idea, again counting the finite number of perfectly distinct (mutually orthogonal) states that can be constructed with given macroscopic constraints. This is the same as the effective dimensionality of the Hilbert space, since a basis is a set of mutually orthogonal states.

The viewpoint that quantum systems generalize classical finite-state ones is familiar if you've thought about quantum computers, which include classical computation as a special case. A more traditional view might be that the finite count of distinct states is an invariant that survives decoherence, which creates a classical probability distribution over a particular finite set of distinct quantum states defined by the decoherence process.