5
$\begingroup$

Today’s exciting press release from Tevatron on the Higgs boson keeps its head cool and say that physicists saw a “hint” of the Higgs boson because the signal is barely above the two-sigma level. In an effort to explain the reasons for this, the press release says:

Physicists claim evidence of a new particle only if the probability that the data could be due to a statistical fluctuation is less than 1 in 740, or three sigmas. A discovery is claimed only if that probability is less than 1 in 3.5 million, or five sigmas.

I wonder: what's the difference between “claim[ing] evidence of a new particle” and “a discovery” (of said new particle)? Is this the common vocabulary for these levels (three and five sigmas) of signal?

The reason I ask is that I can understand the need for having different vocabulary to talk about discoveries with different levels of certitude, and I could see, say, the need for using “discovering a particle” and “confirming the discovery of a particle”, but “evidence” and “discovery” seem semantically very close to me.


Edit — to be clearer, the question is: how universal are the choices of terms (“evidence”, “discovery”) and associated incertitudes?

$\endgroup$
3
  • $\begingroup$ If that question about terminology is off-topic, please feel free to close it, of course! $\endgroup$
    – F'x
    Mar 7, 2012 at 17:49
  • $\begingroup$ I'd say 'evidence' means a bump in the data that is debatable; 'discovery' is something that is extremely certain. Statistical meanings of 'debatable' and 'certain' most probably have to do with standard deviations as mentioned. Though if I were you I'd wait for @annav 's answer. $\endgroup$ Mar 7, 2012 at 17:53
  • 3
    $\begingroup$ @F'x: there was a time when this sort of thing might have been considered off topic, but the scope of the site has been getting broader over time and I think it's a fine (and important!) question. $\endgroup$
    – David Z
    Mar 7, 2012 at 23:32

2 Answers 2

3
$\begingroup$

Evidence and Discovery are both meant in a probabilistic way. Although physics is commonly known as an "exact" science, you always have uncertainty in your measurements due to human errors and/or the measurements precision. Having said that, physicists in this area have ARBITRARILY chosen to accept a new discovery if and only if the probability of random noise in the data is extremely small (but never zero), and they have also chosen to accept that there is evidence of something if the probability of random noise is RELATIVELY small but is not sufficient to claim truthfullness with a strict arbitrary criterion of whats a discovery.

Edit: choice of terminology for evidence and discovery depends on the specific field or organization. One example outside physics is in econometrics where they ususally seek a significance of 0.1 or 0.05 to determine evidence in data. Maybe you can post your question in Cross Validated StackExchange as a Community wiki.

$\endgroup$
3
  • 1
    $\begingroup$ It's not entirely arbitrary, but is related to the embarrassingly many false "discoveries" that where made in the days of the particle zoo (mostly the 60s and 70s). $\endgroup$ Mar 8, 2012 at 3:59
  • 1
    $\begingroup$ Arbitrary in the sense of why 5 sigma and not a little more or a little less to claim a discovery. 5 sigma is arbitrary but is a good rule of thumb. $\endgroup$
    – aortizmena
    Mar 8, 2012 at 5:07
  • $\begingroup$ The existence of a "discovery" threshold is not so arbitrary, but the value that this threshold is set at is arbitrary. Why should a 4.99 sigma result not be a discovery when a 5.00 sigma result would be? Because we said so. $\endgroup$ Jul 5, 2016 at 16:26
3
$\begingroup$

A discovery is evidence beyond doubt, in the sense that it is very, very unlikely to be revoked by further collection of data of the same kind.

Mere evidence is a conspicuous deviation from expectations, but the amount of data collected is too low, so that the event is not yet statistically significant with a high confidence level.

The message you quoted gives precise definitions for the thresholds used by the authors of the message: You need a deviation by 3 standard deviations to claim evidence, but one of 5 standard deviations to claim discovery. Based on the law of large numbers with its $N^{-1/2}$ dependence, one would conclude that it takes a factor of $(5/3)^2\approx 2.8$ more data to reach the discovery level, if the event is indeed real. (The associated probabilities are somewhat dubious, however, being based on the assumption that the deviations are normally distributed, which may or may not be the case.)

I don't know how universally the language is tied to the particular thresholds; it may be a CERN convention, but I doubt that it is a convention universally used in particle physics.

$\endgroup$
3
  • 2
    $\begingroup$ $5\sigma$ is pretty universal as far as I'm aware. $\endgroup$
    – David Z
    Mar 7, 2012 at 21:14
  • $\begingroup$ Is there a widely known document defining the usage? I looked at the web site of the particle data group, but couldn't find out how to search there for such things. $\endgroup$ Mar 7, 2012 at 21:22
  • 1
    $\begingroup$ Not that I know of; it's just one of those things you pick up in the process of learning how experimental particle physics works. After all, this is a convention that basically emerged "organically," by consensus of the global HEP community, not something that was imposed by a standards organization or anything like that. It's possible that these terms might be explained in some textbooks, but I don't know of any specific examples. $\endgroup$
    – David Z
    Mar 7, 2012 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.