# Interpreting the Partition Function and Free Energy Mathematically

Given that

The partition function in statistical mechanics tells us the number of quantum states of a system that are thermally accessible at a given temperature

http://vallance.chem.ox.ac.uk/pdfs/Equipartition.pdf

How does one interpret the statistical mechanical partition function and free energy mathematically, in terms of the sample space, analogous to the way you can interpret microstates and macrostates in terms of a probability sample space? It seems very Shannon's entropy-like. If you take the example of flipping 2 coins:

• Random experiment: Toss two coins
• Example of an Outcome: $10 = (Heads, Tails)$
• Sample space: $S = {11,10,01,00}$, $|S| = 4$
• Examples of Events: 2 Heads $= 2H = \{11\}$, $|2H| = 1$, $1H = \{10,01\}$, $|1H| = 2$, $0H = \{00\}$, $|0H| = 1$

we can translate this into the language of statistical mechanics:

• A microstate is an element of the sample space, e.g. $10$ or $01$.
• A macrostate is an event (a subset of the sample space), e.g. $1H = \{10,01\}$.
• The statistical weight (statistical probability) of a macrostate is the cardinality of the event, e.g. $|1H| = 2$.
• The equilibrium distribution is the most likely macrostate which is the macrostate with the highest statistical weight which is the event with the highest cardinality, e.g. $1H = \{10,01\}$ since $|1H| = 2$.

Finally, the Maxwell-Boltzmann distribution function $n_i$ for the coin toss is found by extremizing $$w(n) = "number \ of \ heads \ in \ n" = \tfrac{2!}{n!(2-n)!}= \tfrac{2!}{n_1!n_2!}$$ with respect to $n_i$ given the constraint equation $n_1 + n_2 = 2$, showing $n_1 = e^0 = 1$ maximizes $w$, i.e. $w(1) = |1H| = |\{10,01\}| = 2$ is the maximum. As fas as I can see, the MB distribution is a function of the total number of particles, $2$, but not the energy as there is no energy in this example.

Here we see everything interpreted mathematically, however I don't see how one does this for the partition function or for the free energy. So to ask my question:

What exactly is the partition function in this example, and what is it's meaning in general?

It sounds like it tells you how many elements are in an event for a given energy and given particle number, i.e. the cardinality of a subset of the sample space which varies as a function of the Lagrange multipliers.

What is the free energy in this example, and in general?

As a concept it seems very similar to the partition function, only it tells you how energies and particle numbers are distributed over all subsets of the sample space that we are considering, not just the most probable one, e.g. it says something about subset 0, subset 1 and subset 2. Though I'm not sure whether it works like this or whether it just relates to the most probable distribution the whole time, and says something about energies over all possible distributions (i.e. for 2, 3, 4, ... as particle numbers in the numerator of the MB distribution function given above).

Equilibrium statistical mechanics is not really about the partition function per se, it just aims at finding (according to one interpretation at least) what is the least biased probability distribution (for the microstates) that satisfies the known constraints on the system.

In your case, you assumed that the probability to get either head or tail for each coin is 1/2 which implicitely uses a statistical inference reasoning to get the probability a priori (for instance Pascal's principle of indifference).

In my view, stat. mech. is concerned with finding those probabilities a priori for each statistical ensemble (microcanonical, canonical, grand canonical and so on). Once this is done, you just apply the usual tools of probability theory to compute averages, variances etc..

In particular, the partition function is closely related to the probability generating function in mathematics.

Let us look at the canonical ensemble describing a statistical ensemble where a system has a conserved volume, number of particles and is in contact with a thermostat at inverse temperature $\beta$.

Equilibrium statistical mechanics tells us that the probability for the system to be in a given microstate $m$ is:

$p(m|\beta) = \frac{e^{-\beta E_m}}{Z(\beta)}$

where normalization of this probability yields:

$\sum_m p(m|\beta) = 1 \: \Rightarrow \: Z(\beta) = \sum_m e^{-\beta E_m}$

Let us now consider a macrovariable $X(m)$ which is a function of the microstate $m$. Its probability distribution can be seen as the marginal distribution of the microstate probability distribution. If the random variables are discrete for simplicity:

$p(X=x|\beta) = \sum_m p(m|\beta) \delta_{X(m),x}$ where $\delta_{x,y}$ is a Kronecker delta function which is one only when $x=y$.

From that point on we can compute the moments for $X$ by using either the probability generating function $G(z) \equiv \mathbb{E}_X[z^X]$ or the moment generating function $M(t) \equiv \mathbb{E}_X[e^{tX}]$ from which we can the moments bt taking derivatives with respect to $z$ or $t$ when the latter are evaluated at $1$ and $0$ respectively.

When $X$ is the energy itself, then the sum over microstates having an energy $E$ can be factorized by $e^{-\beta E}$ and we get:

$p(E=e|\beta) = \frac{g(e)e^{-\beta e}}{Z(\beta)}$

Because both $g(e)$ and $e$ are assumed independent of $\beta$, we can actually notice that all the moments:

$\mathbb{E}_E[E^n] \equiv \sum_e \:e^n \frac{g(e)e^{-\beta e}}{Z(\beta)}$ can be gotten by looking at the quantity $\frac{(-1)^n}{Z(\beta)}\frac{\partial^n Z(\beta)}{\partial \beta^n}$ which is very similar to what we would have done with either the PGF or the MGF or even the CGF.

The reason why, when it comes to moments of the energy variable, we prefer doing this way instead of using the other generating function tools is first because we can (thanks to the fact that both $g(e)$ and $e$ do not depend on $\beta$ or at least we assume so) and second because there is a link with derivatives of the free energy in thermodynamics and energy fluctuations.

Otherwise, in general, it is quite common to introduce an "external field" coupled linearly with the variable $X$ in the energy of the system (which is almost always possible if we imagine the field strength small enough) and that plays the same role as the variable $t$ in the MGF or the CGF approaches. In magnetic systems for instance, it is very common to introduce an external magnetic field to get the moments of the magnetization.

The partition function associated to a random experiment of measuring energy using a random variable $E$ is the moment-generating function for the probability distribution associated with the random variable $E$ (energy):

$$M_E(-\beta) = E[e^{-\beta E}] = \int_{\infty}^{\infty}e^{-\beta E}g(E)dE = \sum_i e^{-\beta E_i}g(E_i) = Z(- \beta)$$

where the equality is either continuous or discrete depending on what you're doing.

I think the moment-generating function arises simply because it uniquely determines the probability distribution associated with a random variable.

The Helmholtz free energy associated to a random experiment of measuring energy using a random variable $E$ is the the cumulative generating function per unit variable:

$G_E(-\beta) = \frac{\ln[M_E(-\beta)]}{- \beta}= \frac{\ln Z(- \beta)}{- \beta}$

As determining the Taylor coefficients of the Cumulant generating function uniquely determines the coefficients of the moment generating function the CGF also uniquely determines the probability distribution function associated with a random variable.

The only intuition I have so far is that the energy random variable for an entire macroscopic system can be thought of as a sum of independent random variables for the macroscopic subsystems, i.e. the macrostates, and as the moment generating function is multiplicative the cumulant generating function is additive so we can apply Noether's theorem and associate integrals of motion to the Free Energy, thus allowing us to get other quantities associated to the system.

• Note that although the partition function shares many things with the generating function tool, it is in essence a different object. That is because the probability distribution for $E$ is not simply $g(E)$ but $g(E)e^{-\beta E}$ and therefore it cannot directly been "named" as a generating function. To give you another point of view, the partition function can be thought of as a Laplace transform of the density of state $g(E)$ and the derivative<->multiplication properties of this object leads to the generating function-like behaviour I was talking about. May 5, 2014 at 20:22
• I don't think that's correct, if there were true I would have to have written $$M_E(-\beta) = E[e^{-\beta E}] = \int_{\infty}^{\infty}e^{-\beta E} \cdot (e^{-\beta E}g(E))dE = \sum_i e^{-2 \beta E_i}g(E_i)$$ above and I'd get incorrect answers. May 5, 2014 at 21:18
• No, according to the actual definition you would have $M_{\beta}(z) \equiv E[z^E] = \frac{1}{Z(\beta)}\int_0^{+\infty} g(E)e^{-\beta E}z^E \: dE$. And hence all the moments can be obtained by taking derivatives of this thing with respect to $z$ at the point where $z=1$. This would be perfectly fine, it is just that the same trick can be done directly with the sum over the probability weights with derivatives with respect to $\beta$ and evaluated at $\beta$. May 6, 2014 at 7:26
• So when I said it was closely related to the generating function, it was more to say that it has essentially all its advantages/properties even if (most of the time) it doesn't use an extra variable $z$ to do the job because it is not needed...for the energy that is. You would however need to use an extra variable $z$ or rather we prefer using the concept of an "external field" to get the moments of any other macrovariable. As an example in statistical mechanics of magnetic systems, we always add an external magnetic field with respect to which we can take derivatives and get the magnetization May 6, 2014 at 7:39
• I have edited my answer, feel free to comment on it. May 6, 2014 at 14:06