I am a computer science master student. In a statistical learning theory course I am taking, mean field approximation was introduced to approximately solve non-factorizable Gibbs distributions that were derived using maximum entropy inference. Our professor has a strong background in physics and often uses terms from statistical physics. Unfortunately, I lack that background. So,

  1. Are there any resources explaining mean field approximations from a non-physical/computer science perspective? I couldn't find any.

  2. Or alternatively, are there "crash course"-like resources that would allow me to understand one of the more physically motivated explanations without looking up tons of terms?

EDIT: Here are some details that hopefully help to narrow it down: In this course, we are trying to sample from intractable, non-factorizable Gibbs-distributions (mainly in the context of clustering). Apparently, this cannot be done (efficiently), therefore we retreat to a mean-field approximation.


Before answering, please see our policy on resource recommendation questions. Please write substantial answers that detail the style, content, and prerequisites of the book, paper or other resource. Explain the nature of the resource so that readers can decide which one is best suited for them rather than relying on the opinions of others. Answers containing only a reference to a book or paper will be removed!

migrated from stats.stackexchange.com May 22 '12 at 13:24

This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.

  • $\begingroup$ There is a better site for that. $\endgroup$ – user68 May 22 '12 at 13:24
  • 1
    $\begingroup$ Your best bet is to ask your professor for more details, but actually aside from its historical development within physics, mean-field approximations basically are defined by replacing non-factorizable probability distributions with factorizable ones that have some correlations put in by hand (the "mean fields" or "Weiss fields" of stat.phys.) The extra correlations should be predicted by the distribution, though, so they are calculated self-consistently in the factorizable one. $\endgroup$ – wsc May 22 '12 at 13:51
  • $\begingroup$ Try > W. D. McComb, Renormalization Methods: a guide for beginners, 2004. The first three chapters give you what you want. $\endgroup$ – user9365 May 22 '12 at 20:45
  • $\begingroup$ @Mbq: Given that the OP wants a "non-physical" description this might have been better moved to Computation Science, but that's a beta site. $\endgroup$ – dmckee May 22 '12 at 21:03
  • $\begingroup$ @dmckee You're right, I've... missed. $\endgroup$ – user68 May 22 '12 at 22:14

A concise 16 page exposition (starting essentially from scratch) is given in Sections 8.1-8.3 of my book

Classical and Quantum Mechanics via Lie algebras http://lanl.arxiv.org/pdf/0810.1019v2.pdf

The mean field theory (and corrections to it) appears at the very end of Section 8.3.

In your case, the algebra is commutative, so you don't need to understand all the fine points related to noncommutative observables in the quantum case, and the mean field will simply be a high-dimensional Gaussian, where the cumulant generating function $W(f)$ is just the exponential of a quadratic form.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.