# Generalized Entropy as a starting point for Statistical Mechanics

In introductory courses on Statistical Mechanics, much is derived from using the generalized entropy of some system (such as the canonical partition function etc):
$$\tilde{S} = k_B \sum_{n}p_n\ln(p_n)$$ However, not much insight is usually given into why this is entropy is the case (such as how we can see the correspondence between Entropy in Thermodynamics) which would appear to be useful. How can one either derive or come to an appreciation of this equation?

• Do you mean the proving uniqueness ( up to the constant) of Shannon entropy subject to some additive and montonicity assumptions? – AHusain Sep 22 '16 at 6:31
• @AHusain I'm sure most reasonable readers would take that to be one of many excellent and intellectually fulfilling motivations for the use of Shannon's notion, which, IMO, is as good as it gets in physics. A concise, readable account of this approach is section 2 in E. T. Jaynes, "Information Theory and Statistical Mechanics", Phys. Rev. 1957 – WetSavannaAnimal Sep 22 '16 at 7:41

Now consider the Boltzmann distribution $p(\omega) = Z^{-1} e^{-\beta H(\omega)/k_B}$ (suppose we are working on a finite state space). The entropy of $p$ is \begin{align} \tilde S &= -k_B \sum_\omega p(\omega) \log p(\omega) \\ &= -k_B \sum_\omega Z^{-1} e^{-\beta H(\omega)/k_B}(-\beta H(\omega)/k_B - \log Z)\\ &= \beta \langle H \rangle + k_B \log Z. \end{align} Re-arranging this gives \begin{equation} -\beta^{-1} k_B \log Z = \langle H \rangle - \beta^{-1} \tilde S, \end{equation} relating free energy ($-\beta^{-1} k_B \log Z$), internal energy ($\langle H \rangle$), and entropy.