# Derivation of (Shannon) entropy in the canonical ensemble

I am studying Statistical Mechanics and I cannot understand how to find the formula for the entropy $$S = -k_B\sum_n p(n) \ln p(n)$$ which coincides with the Shannon entropy.

I have red many books and bibliography on the subject. The one I like more are the lectures of David Tong. In page 22 he says to consider a huge number of copies of the system+reservoir (so this copies form a microcanonical ensemble) and to use this in conjunction with the definition of entropy $S = -k_B \ln \Omega$. What I don't understand is what is the $\Omega$ that he calculates, i.e. what is $$\Omega = \frac{W!}{\prod_n (p(n)W)!}$$

I appreciate any help, be it an explanation or some alternative bibliography on the subject.

• It is clearly stated in the text: "We must only figure out how many ways there are of putting $p(n)W$ systems into state $n$ for each $n$. That’s a simple combinatoric problem: the answer is $\Omega=\dots$" What is exactly that you don't understand here? – valerio Feb 6 '18 at 7:41
• You may find the multinomial coefficients useful in seeing where the expression comes from – By Symmetry Feb 6 '18 at 10:10
• You realise, I expect, that $p(n)W$ is the number of systems in the $n^\text{th}$ state in the ensemble of $W$ systems? [I'm slightly surprised that this is about a $micro$canonical ensemble, but I don't have access to Tong.] – Philip Wood Feb 6 '18 at 10:13
• @PhilipWood You can access the pdf from the link given by OP. – valerio Feb 6 '18 at 11:41