# What does a correlation function measure and how does it do this mathematically?

I would really appreciate if someone could explain.

What does a correlation function like a density-density correlation function $$C_{nn}(\vec x_1, \vec x_2)= \langle n(\vec x_1) n(\vec x_2)\rangle$$ measure, or what does it mean as a function? By this i don't mean what the interpretation of the result is but what does it do mathematically?

And even more important how does it do this. If someone could explain this in general terms general functions of randomly variables it would be very helpfull.

I know about this question but none of the answeres really describes what generally a correlation function signifies or how it does this mathematically. But this is exactly what i need to know. Shurely there must be a way to proove or at least understand this completely theoretically.

I know that the correlation is defined as covariance function normalized as in wikipedia. The normalization seemes to always be missing in physics. Is this because we just use normalized random variables?

• I think Lubos' answer there answers this question. Could you elaborate what dissatisfies you about it? – ACuriousMind Mar 1 '15 at 15:45
• shure. I know that the correlation is defined as covariance function normalized as in wikipedia. The normalization seemes to alwys be missing in physics. In qm the normalisation is possibly in the states, I don't know but either way i have never seen this addressed. Secondly i read in many places "It will have value 0 when the covariance is zero and value 1" but why this is i don't know. Shurely there must be a way to proove or at least understand this theoretically. The expression in the covariance $cov(X,Y)=E(XY)-E(X)E(Y)$ achives this, but how or why? I hope this is more helpful. – pindakaas Mar 1 '15 at 16:04
• As I understand it now, it seems that you are asking more about what the covariance measures? And what "correlation" means in general? Then this might be better suited for mathematics. – Martin Mar 2 '15 at 9:07
• read the wikipedia definition it uses the covariance. The definition every physicist uses is very different from this. So why is that? – pindakaas Mar 2 '15 at 9:13
• I venture physicists just take the variables to be normalized? can any one at least confirm or deny that suspicion or point me to some source on this? – pindakaas Mar 2 '15 at 18:34

The correlation function measures, as you would expect, how correlated two random variables are. That is, how often two random variables have similar values.

We can construct such a function very simply. Say you are flipping coins, and you want to know if their results are correlated. To quantify things, call "heads" $+1$ and "tails" $-1$. To make things concrete, lets say the results from flipping each coin 5 times are: \begin{align} \text{Coin } A:&\ +, -, -, +, - \\ \text{Coin } B:&\ -, -, -, -, + \\ \text{Coin } C:&\ +, -, +, +, - \\ \text{Coin } D:&\ -, +, -, -, + \\ \end{align} To measure how "correlated" the a pair of results are, we want a function that is positive when the two results are similar, and negative when they are dissimilar. The easiest function is multiplication: it will be positive when the coins have the same result (++ or --) and negative when they differ (+- and -+). We can multiply each trial, then average the results to get an overall estimate of how similar the results are: \begin{align} C_{AB} &= \frac{-1+1+1-1-1}{5} = 0.2 \\ C_{AC} &= \frac{+1+1-1+1+1}{5} = 0.6 \\ C_{AD} &= \frac{-1-1+1-1-1}{5} = -0.6 \\ \end{align} So $A$ and $B$ seem pretty uncorrelated, $A$ and $C$ may be correlated, and $A$ and $D$ are anti-correlated.

The principle is exactly the same in statistical mechanics. You have two sets of random variables, the density at $x_1$ and the density at $x_2$. The variables are random, because you don't know what the exact density field $n(x)$ is, you only have a probability distribution (e.g. Boltzmann) over many possible $n(x)$'s. To calculate the correlation of $n(x_1)$ and $n(x_2)$, multiply them for every realization of $n(x)$ and average the results. We write this as: $$C_{nn}(x_1, x_2) = \langle n(x_1) n(x_2)\rangle$$

Now, you mentioned the covariance in your comment. It could be that for your problem $C_nn(x_1, x_2)$ does not give you the information you want. Perhaps $\langle n(x_1)\rangle$ and $\langle n(x_2)\rangle$ are different, but both large and positive. Then the average you do in $C_{nn}$ will be dominated by just multiplying the average density of both fields. In cases like this, you may be interested not in the correlation of $n(x_1)$ and $n(x_2)$ themselves, but in correlations of their fluctuations from equilibrium: $n(x) - \langle n(x)\rangle$. This correlation function is the covariance! \begin{align} C_{n-\bar{n},n-\bar{n}}(x_1, x_2) &= \Big \langle \big (n(x_1)-\langle n(x_1) \rangle \big ) \big (n(x_2)-\langle n(x_2) \rangle \big ) \Big \rangle \\ &= \Big \langle n(x_1)n(x_2) \Big \rangle - \Big \langle n(x_1)\langle n(x_2)\rangle \Big \rangle - \Big \langle \langle n(x_1) \rangle n(x_2) \Big \rangle + \langle n(x_1)\rangle \langle n(x_2) \rangle \\ &= \Big \langle n(x_1)n(x_2) \Big \rangle - \Big \langle n(x_1)\Big \rangle \Big \langle n(x_2) \Big \rangle \\ &= Cov(n(x_1),n(x_2)) \end{align}

• Thanks for the answer. But it leaves some questions still. The Correlation is defined as the covariance divided by the expectation value of the individual variables and i still have no idea how this relates to the object physicists use. Secondly how does multiplying two variables and averaging the values in general how much they are the same? I tried what you did in your example for a dice throw and i have no idea how this would generalise to this, or even something like a density function, because it is again just a specific example. – pindakaas Mar 2 '15 at 8:15
• Dividing by the product of the individual expectation values is a common way of normalizing the correlation function. It gives you a handy way of comparing correlations for different functions. For instance, comparing the density-density correlation to the velocity-velocity correlation. Generically this is difficult because the units are different, normalizing makes the function dimensionless and easy to compare. – Geoff Ryan Mar 2 '15 at 8:21
• I know that. That was not the point of the comment – pindakaas Mar 2 '15 at 8:25
• Averaging the product of your random variables will tell you how often they are both positive or negative. In the case of rolling dice (or density), since the random numbers are always positive the correlation function I defined about will always be positive too. This may not be very useful, so the Covariance may make more intuitive sense in this case. – Geoff Ryan Mar 2 '15 at 8:25
• Sorry, perhaps I misunderstood. Could you rephrase your question? – Geoff Ryan Mar 2 '15 at 8:29

I had the same question and did not find a totally satisfactory answer. However, my guess is that in Physics, theory is almost always dealing with normalized variables, as equations need to be exact and probabilities are always normalized. Covariance is a practical approximation of correlation functions that generalizes the concept to real measurements that are not necessarily normalized. For getting practical bounded correlation values (between -1 and 1), the covariance is divided by variances of each variable. In Physics many "states" are just defined as binary 0 or 1, or -1,+1 (and in some probabilistic examples, such as the dice or coin tosses) which make the averaging process already normalized between 0 and 1. Note that averaging is the same as integrating using a probability density (the case of the dice, the density is constant equals 1/6), so this could also explain why in physics these correlations are bounded. However, I am not sure this means that the correlation in physics is the same as mathematical statistical correlation, which is also normalized by the variance of the variables. I think they are two different magnitudes and this is just a problem of terminology.