# Reference: authoritative reference on Gibbs and Boltzmann's entropy

Can someone reference a good, standard textbook on thermodynamics or quantum mechanics that explicitly states the formula for Gibbs and Boltzmann's entropy (or maybe Shannon as well)?

I am asking because I have seen many papers where authors use these terms interrchangably and sometimes will even say Gibbs-Boltzmann entropy so I have became very confused.

• Wrong place to ask... we recently had a post here that ended up with most people confusing these concepts, too. :-) Having said that, what's wrong with the Wikipedia entry: en.wikipedia.org/wiki/Boltzmann%27s_entropy_formula. As for Shannon... that's got nothing to do with physics proper, but you will find a lot of folks who don't grok why it's different. Commented Jul 3, 2016 at 22:42
• Cédric Villani has a very useful and physically intuitive book on entropy/irreversibility. That's about as fundamental as you will probably get... Commented Jul 5, 2016 at 1:13
• I'm not sure if I can post this as an answer since it is by no means authoritative, but a few years ago I gave a seminar (as a student) on exactly this topic: the different notions of entropy, in particular defining and explaining Boltzmann, Gibbs and Shannon entropy, and their differences. I just uploaded my seminar notes: microcanonical.com/notes_entropy_seminar.pdf I hope it's useful! I know how frustrating the literature can be on this topic. Commented Jul 10, 2016 at 12:59

## Gibbs vs Boltzmann entropies

The question was discussed by Jaynes in Gibbs vs Boltzmann entropies. American Journal of Physics, 33(5):391–398, 1965 (also availableble from here as Ref 21).

One must be careful by what we mean "Boltzmann entropy" because Boltzmann tried a lot of different things and came across various different aspects of entropy.

Boltzmann entropies

• Perhaps the most well known Boltzmann entropy is the equation $$\tag{1} S = k \log W$$ which is inscribed on his tomb. Today we understand this to represent the entropy of an isolated system with fixed energy, volume, and number of particles.

• Boltzmann also came up with the expression $$\tag{2} S = -k \sum_i p_i \log p_i$$ by discretizing the phase space of particles.

• Boltzmann obtained yet another entropy via his $$H$$ theorem: $$\tag{3} S = - \int f(v) \log f(v) dv$$ where $$f(v)$$ is the velocity distribution of a single particle in a dilute gas. As Jaynes shows, this entropy converges to the Gibbs entropy in the ideal-gas limit.

Gibbs entropy Gibbs wrote the following equation for the entropy: $$\tag{4} \boxed{ S = - k\int \rho(\Gamma) \log \rho(\Gamma) d\Gamma }$$ where $$\Gamma = (\mathbf r_1,\cdots;\mathbf q_1,\cdots)$$ is the vector of positions and momenta of all particles and $$\rho(\Gamma) d\Gamma$$ is the $$N$$-particle probability, namely, the probability to find the microstate of the entire system in the the region $$(\Gamma,\Gamma+d\Gamma)$$ of phase space. This is the entropy of statistical mechanics.

Shannon entropy Shannon obtained the entropy of generic probability distribution $$p_i$$ as $$\tag{5} S = - \sum_i p_i \log p_i$$

Relationship between the various entropies

• Gibbs's entropy is Shannon's entropy in the continuous domain applied to the equilibrium distribution of microstates.

• Boltzmann's entropy in Eq 1 is obtained from Gibbs entropy when all microstates are equally probable (the case in an isolated system, i.e., a microcanonical ensemble).

• Boltzmann's Eq 2 is analogous to the Gibbs/Shannon entropy.

• Boltzmann's entropy in Eq 3 represents thermodynamic entropy only in the ideal-gas limit (no inter-molecular forces, i.e., no correlation between states of particles).