I couldn't find any publication by LIGO that explains how we should interpret this value. The closest I have found is the following quote:
This means that a noise event mimicking GW150914 would be exceedingly rare - indeed we expect an event as strong as GW150914 to appear by chance only once in about 200,000 years of such data! This false alarm rate can be translated into a number of "sigma" (denoted by s), which is commonly used in statistical analysis to measure the significance of a detection claim. This search identifies GW150914 as a real event, with a significance of more than 5 sigma.
http://www.ligo.org/science/Publication-GW150914/index.php
From my reading, it appears that $5.1\sigma$ significance refers to:
The probability of observing such a signal given that the model of background noise correctly describes all input to the detectors at the time of the signal.
I would like to verify that the above interpretation is correct and is different from the probability GW150914:
- arose due to chance
- was caused by a gravitational wave
- was caused by a BH-BH merger
I ask because I have seen posts on this site and elsewhere (both news and blogs) that seem to imply differently. I worry I may be misunderstanding some terminology specific to astrophysics.
Also, does anyone know what calculations were used to convert false alarm rate to # of sigmas? This detail seems to have been left out of the papers, so I assume it is something trivial I am missing due to lack of background in this area.
Edit:
Let me clarify (what I have learned is an incorrect) interpretation #1 above. This is Bayes' rule:
$
p(H|O)=\frac{p(H)p(O|H)}{p(O)} \tag{1}
$
where,
$$H=\text{Hypothesis (model of background noise describes}\\ \text{all input to the detectors at the time of the signal)}$$
$$O= \text{Observation (the GW150914 signal)}$$
Just to be 100% clear:
$$
p(H|O)=\text{The probability H is true given O has been observed}
\\
p(O|H)=\text{The probability of observing O given H is true}
\\
p(H)=\text{The probability H is true} \textit{ independent } \text{of observation O}
\\
p(O)=\text{The probability of observing O} \textit{ independent } \text{of whether H is true}
$$
The last term can be rewritten as:
$$
p(O)= p(H)p(O|H)+p(\neg H)p(O|\neg H) \tag{2}
$$
where the probability H is false is denoted by
$$p(\neg H)=1-p(H)\tag{3}$$
In the answers, we established the $\sigma$-level is a simple transformation of the p-value, which equals $p(O|H)$. It is clear that $p(H|O)$ must have a different numerical value than the p-value except under some very specific circumstances, i.e. when $p(H)=p(O)$. The p-value is calculated under the assumption that $H$ is true, and from equations 1/2/3 we see that $p(H|O)$ explicitly depends on both $p(H)$ and the probability of observing such a signal if $H$ is false: $p(O|\neg H)$.
If our hypothesis is true, I think we all agree the only way to get a signal like GW150914 is a chance coincidence of noise patterns between the two LIGO detectors. So when writing we often use shorthand such as: $$H=\text{any signal is due to, i.e. caused by, chance coincidence}$$ or $$H=\text{any signal is not real}$$
There are many shorthand ways of saying the same thing that confuses things. The point is that the p-value is not the probability GW150914 was caused by (arose from; is due to) chance (background noise; random coincidence). It is also not the probability GW150914 "isn't real", or "how unlikely" it is that GW150914 is due to chance.
In this case, the p-value is apparently $p(O|H)\approx2\times10^{-7}$. Also, apparently the only other plausible explanation is a BH-BH merger. In an earlier question we estimated the prior probability of this to be $\approx10^{-4} \text{ to } 10^{-1}$. If we suppose that is the only other possible explanation, that must be the probability that H is false independent of observing GW150914: $p(\neg H)$.
First, lets use the lower bound: $p(\neg H)\approx10^{-4}$. From equation 3, then $p(H)\approx0.9999$. Also, GW150914 apparently matched prediction exactly. Therefore, probability of seeing such a signal given that H is false is $p(O|\neg H)\approx1$. Plugging in these values we get:
$$p(H|O)=\frac{0.9999\times2\times10^{-7}}{0.9999\times2\times10^{-7}+10^{-4}\times1}\approx0.002$$
Doing the same for the upper bound I get $p(H|O)\approx 1.8\times10^{-6}$. Now we can say "the probability GW150914 occurred due to chance ranges from $2\times10^{-3} \text{ to } 1.8\times10^{-6}$," which is quite different from the p-value. Any mistakes in this reasoning?
1. arose due to chance
? $\endgroup$the probability GW150914: p(GW150914 arose due to chance|data)
, ie the probability of a probability. Also, thanks both answers address my first concern. Can someone give the exact calculation of getting from false alarm rate$=1/200,000$ years to $5.1\sigma$? $\endgroup$