Interpreting the Partition Function and Free Energy Mathematically

Question

Given that

The partition function in statistical mechanics tells us the number of quantum states of a system that are thermally accessible at a given temperature

http://vallance.chem.ox.ac.uk/pdfs/Equipartition.pdf

How does one interpret the statistical mechanical partition function and free energy mathematically, in terms of the sample space, analogous to the way you can interpret microstates and macrostates in terms of a probability sample space? It seems very Shannon's entropy-like. If you take the example of flipping 2 coins:

• Random experiment: Toss two coins
• Example of an Outcome: $10 = (Heads, Tails)$
• Sample space: $S = {11,10,01,00}$, $|S| = 4$
• Examples of Events: 2 Heads $= 2H = \{11\}$, $|2H| = 1$, $1H = \{10,01\}$, $|1H| = 2$, $0H = \{00\}$, $|0H| = 1$

we can translate this into the language of statistical mechanics:

• A microstate is an element of the sample space, e.g. $10$ or $01$.
• A macrostate is an event (a subset of the sample space), e.g. $1H = \{10,01\}$.
• The statistical weight (statistical probability) of a macrostate is the cardinality of the event, e.g. $|1H| = 2$.
• The equilibrium distribution is the most likely macrostate which is the macrostate with the highest statistical weight which is the event with the highest cardinality, e.g. $1H = \{10,01\}$ since $|1H| = 2$.

Finally, the Maxwell-Boltzmann distribution function $n_i$ for the coin toss is found by extremizing $$w(n) = "number \ of \ heads \ in \ n" = \tfrac{2!}{n!(2-n)!}= \tfrac{2!}{n_1!n_2!}$$ with respect to $n_i$ given the constraint equation $n_1 + n_2 = 2$, showing $n_1 = e^0 = 1$ maximizes $w$, i.e. $w(1) = |1H| = |\{10,01\}| = 2$ is the maximum. As fas as I can see, the MB distribution is a function of the total number of particles, $2$, but not the energy as there is no energy in this example.

Here we see everything interpreted mathematically, however I don't see how one does this for the partition function or for the free energy. So to ask my question:

What exactly is the partition function in this example, and what is it's meaning in general?

It sounds like it tells you how many elements are in an event for a given energy and given particle number, i.e. the cardinality of a subset of the sample space which varies as a function of the Lagrange multipliers.

What is the free energy in this example, and in general?

As a concept it seems very similar to the partition function, only it tells you how energies and particle numbers are distributed over all subsets of the sample space that we are considering, not just the most probable one, e.g. it says something about subset 0, subset 1 and subset 2. Though I'm not sure whether it works like this or whether it just relates to the most probable distribution the whole time, and says something about energies over all possible distributions (i.e. for 2, 3, 4, ... as particle numbers in the numerator of the MB distribution function given above).

Answer 1

Equilibrium statistical mechanics is not really about the partition function per se, it just aims at finding (according to one interpretation at least) what is the least biased probability distribution (for the microstates) that satisfies the known constraints on the system.

In your case, you assumed that the probability to get either head or tail for each coin is 1/2 which implicitely uses a statistical inference reasoning to get the probability a priori (for instance Pascal's principle of indifference).

In my view, stat. mech. is concerned with finding those probabilities a priori for each statistical ensemble (microcanonical, canonical, grand canonical and so on). Once this is done, you just apply the usual tools of probability theory to compute averages, variances etc..

In particular, the partition function is closely related to the probability generating function in mathematics.

Let us look at the canonical ensemble describing a statistical ensemble where a system has a conserved volume, number of particles and is in contact with a thermostat at inverse temperature $\beta$.

Equilibrium statistical mechanics tells us that the probability for the system to be in a given microstate $m$ is:

$p(m|\beta) = \frac{e^{-\beta E_m}}{Z(\beta)}$

where normalization of this probability yields:

$\sum_m p(m|\beta) = 1 \: \Rightarrow \: Z(\beta) = \sum_m e^{-\beta E_m}$

Let us now consider a macrovariable $X(m)$ which is a function of the microstate $m$. Its probability distribution can be seen as the marginal distribution of the microstate probability distribution. If the random variables are discrete for simplicity:

$p(X=x|\beta) = \sum_m p(m|\beta) \delta_{X(m),x}$ where $\delta_{x,y}$ is a Kronecker delta function which is one only when $x=y$.

From that point on we can compute the moments for $X$ by using either the probability generating function $G(z) \equiv \mathbb{E}_X[z^X]$ or the moment generating function $M(t) \equiv \mathbb{E}_X[e^{tX}]$ from which we can the moments bt taking derivatives with respect to $z$ or $t$ when the latter are evaluated at $1$ and $0$ respectively.

When $X$ is the energy itself, then the sum over microstates having an energy $E$ can be factorized by $e^{-\beta E}$ and we get:

$p(E=e|\beta) = \frac{g(e)e^{-\beta e}}{Z(\beta)}$

Because both $g(e)$ and $e$ are assumed independent of $\beta$, we can actually notice that all the moments:

$\mathbb{E}_E[E^n] \equiv \sum_e \:e^n \frac{g(e)e^{-\beta e}}{Z(\beta)}$ can be gotten by looking at the quantity $\frac{(-1)^n}{Z(\beta)}\frac{\partial^n Z(\beta)}{\partial \beta^n}$ which is very similar to what we would have done with either the PGF or the MGF or even the CGF.

The reason why, when it comes to moments of the energy variable, we prefer doing this way instead of using the other generating function tools is first because we can (thanks to the fact that both $g(e)$ and $e$ do not depend on $\beta$ or at least we assume so) and second because there is a link with derivatives of the free energy in thermodynamics and energy fluctuations.

Otherwise, in general, it is quite common to introduce an "external field" coupled linearly with the variable $X$ in the energy of the system (which is almost always possible if we imagine the field strength small enough) and that plays the same role as the variable $t$ in the MGF or the CGF approaches. In magnetic systems for instance, it is very common to introduce an external magnetic field to get the moments of the magnetization.

Answer 2

The partition function associated to a random experiment of measuring energy using a random variable $E$ is the moment-generating function for the probability distribution associated with the random variable $E$ (energy):

$$M_E(-\beta) = E[e^{-\beta E}] = \int_{\infty}^{\infty}e^{-\beta E}g(E)dE = \sum_i e^{-\beta E_i}g(E_i) = Z(- \beta)$$

where the equality is either continuous or discrete depending on what you're doing.

I think the moment-generating function arises simply because it uniquely determines the probability distribution associated with a random variable.

The Helmholtz free energy associated to a random experiment of measuring energy using a random variable $E$ is the the cumulative generating function per unit variable:

$G_E(-\beta) = \frac{\ln[M_E(-\beta)]}{- \beta}= \frac{\ln Z(- \beta)}{- \beta}$

As determining the Taylor coefficients of the Cumulant generating function uniquely determines the coefficients of the moment generating function the CGF also uniquely determines the probability distribution function associated with a random variable.

http://en.wikipedia.org/wiki/Cumulant-generating_function#Relation_to_statistical_physics

The only intuition I have so far is that the energy random variable for an entire macroscopic system can be thought of as a sum of independent random variables for the macroscopic subsystems, i.e. the macrostates, and as the moment generating function is multiplicative the cumulant generating function is additive so we can apply Noether's theorem and associate integrals of motion to the Free Energy, thus allowing us to get other quantities associated to the system.