# How does the "Look Else Where Effect" affect the chances of detecting a false diphoton excess at the LHC?

Back in December 2015, there was found a 750 GeV diphoton excess in both CMS and ATLAS at the same location with a significance well above $3\sigma$; a 0.13% chance of being false.

However, there hasn't been any recent rumours on a well known blog confirming that this diphoton excess is increasing with 2016 data, currently $13.5 fb^{-1}$ from both ATLAS and CMS. In my opinion, it's now looking likely that the 750Gev diphoton excess is disappearing in both detectors, despite both signals being seen at the same location, and having a local 0.13% chance of being false!

The Look Else Where Effect has the effect of making it likely that a statistically insignificant local event is likely to occur somewhere over a large parameter space. I would like to know: what are the chances that a false $3\sigma$ diphoton excess will be detected at the LHC?

• This is, of course, exactly why five sigma is considered a minimum for claiming anything solid from a simple bump-hunting analysis. To answer the question you need to do some Monte Carlo, and you need to know how they typically bin their data. You could look at the CMS data dump from the 2011 run for some hints. Jul 17, 2016 at 22:21
• You are reading the wrong blogs :) Jokes aside, the mentioned one is a very heavily opinion-based blog that a large percentage of physicists view with caution. May 20, 2019 at 14:21

The chances that a false 3-sigma excess is observed in LHC experiments is so high that it happens of order monthly, and indeed these excesses are not usually reported or pointed out as anything of note.

The only thing that can be estimated with some certainty is the look elsewhere effect within a given, well-defined analytical problem. Indeed Monte Carlo simulations are the method of choice. For example, if you have a number of bins in your binned data, you can evaluate how that changes the true (global) probability of observing a 3-sigma excess in any one such bin. However, the truth is much more complicated, because in addition to various bins in a given analysis, there are also different parameters in the same analysis which contribute too. Then not everything would count as an excess because you may have requirements, e.g. from the resolution of your detector, that neighbouring bins show a consistent excess. It gets difficult quickly to model accurately. But then there are also many different kinds of analyses, which each also contribute to the look elsewhere effect that is relevant when assessing something from the news. This is essentially impossible to quantify.

At the end of the day, if you include every plot made by every of the thousands of graduate students on the LHC experiments you can easily see that the true look elsewhere effect is completely unquantifiable. For illustration, the probability of a five-sigma excess is somewhere in the $$10^{-7}$$ ballpark by definition. With, say, 10,000 analysts working on LHC experiments, each making just one new plot a week, and the experiment running for a decade, that's already close to this trial factor. Based on this simple estimate one could expect even a five-sigma excess to appear in an LHC analysis purely as a statistical fluke.

So to answer your question: It is virtually guaranteed that multiple false three sigma excesses will continue to pop up in LHC data.

The exact answer depends on the specifics of the analyses, and also on the overall number of analyses performed.

Speaking however about the first time that the specific diphoton excess was announced by both experiments, you can find here the values for the significance before and after the look elsewhere effect being taken into account (and also links to slides with details).

To quote, the values before/after were 3.6 / 1.9 sigma for ATLAS and 2.6 / 1.2 sigma for CMS.