# Why is (von Neumann) entropy maximized for an ensemble in thermal equilibrium?

Consider a quantum system in thermal equilibrium with a heat bath. In determining the density operator of the system, the usual procedure is to maximize the von Neumann entropy subject to the constraint that the ensemble average of the Hamiltonian has a fixed value.

What justifies this assumption?

Sakurai, in his QM text, writes

To justify this assumption would involve us in a delicate discussion of how equilibrium is established as a result of interactions with the environment.

I'd appreciate if someone could shed some light on this. References are welcome as well.

I've heard the suggestion that a thermal equilibrium ensemble is simply defined by that density operator which solves the constrained optimization problem above. If this is the case, then why are real physical systems that are in weak contact with heat baths for long periods well described by such mathematical ensembles, and how would one identify the Lagrange multiplier $\beta$ that arises with inverse temperature of the heat bath?

-

## 1 Answer

You need to read this paper by Jaynes. I can't explain it as well as him, but I will try to summarise the main points below.

The first thing is to realise that the entropy is observer-dependent: it depends on what information you have access to about the system. A finite temperature means that you don't have access to all the information about the state of the system; in particular, you cannot keep track of the (infinite) degrees of freedom of the bath. However, suppose that some demon could keep track of all the degrees of freedom of the system and bath: he/she sees zero entropy. For the demon, it looks a bit like the total system is at zero temperature (although really it is better to say that temperature is ill-defined for the demon).

Given that you are ignorant (sorry, but at least I'm not calling you a demon), you need to find a consistent prescription for assigning probabilities to the different microstates. The prescription must be 'honest' about what you do or don't know. The entropy is in some sense a unique measure of ignorance, as proved by Shannon. Therefore you should 'maximise your ignorance', subject to the constraint that you do know certain macroscopic observables, e.g. average energy or average particle number if the system is open, etc.

Maximising the entropy of the system is the most logical way to assign probabilities to the microstates of the system, given access only to a limited subset of observables. The same 'MaxEnt' principle is quite general and applies to all statistical analysis, not only physics. The Lagrange multiplier $\beta$ is identified with inverse temperature by comparing the outcome of this abstract procedure to the experimental facts of phenomenological thermodynamics.

If you are interested in the actual dynamics of equilibration, there has been a lot of literature on this recently, especially in mesoscopic systems. Particular focus is laid on the integrability of the system: non-integrable (chaotic) systems do thermalise, whereas there is a fair bit of evidence that integrable systems do not thermalise properly. Intuitively, this is because integrable systems have a maximal set of locally conserved quantities so that, even when in contact with a heat bath, the memory of the initial conditions is never quite lost.

See, for example: Dynamics of thermalisation in small Hubbard-model systems and Thermalization and ergodicity in many-body open quantum systems, if you search 'thermalization' (sic) on arxiv then you will find many more.

-
I think this is a very good question, and it's the subject of a lot of research at the moment, although it comes with a caveat. In the context of classical physics, it's important to remember that the system is always really in a pure state. So the emergence of the Gibbs ensemble cannot really be separated from the observer's subjective knowledge about the system. I think of it as ignorance/information/entropy being created dynamically each time the system interacts with degrees of freedom that the observer cannot track. –  Mark Mitchison Feb 6 '13 at 19:30
(contd.) When the energy/particle etc. distribution is uniform over scales longer than the minimal achievable experimental resolution then further interactions do not increase the observer's entropy, i.e. the probability distribution hits the steady state. –  Mark Mitchison Feb 6 '13 at 19:34
(contd.) However, in quantum mechanics you can't say the system is always really in a pure state, and the uncertainty of the observer is not subjective, it's required by the laws of physics. You can study equilibration of small quantum systems in a heat bath and you find that the system evolves towards a thermal state, basically due to entanglement with the bath. Likewise, it has been shown that almost all bipartite states for systems with a natural system-environment division are locally thermal, see here. –  Mark Mitchison Feb 6 '13 at 19:39
Wow thank you so much Mark. I couldn't have asked for better responses; I'll definitely look at all of the references you suggested. –  joshphysics Feb 6 '13 at 19:43
My pleasure :-) –  Mark Mitchison Feb 6 '13 at 20:58