# What's the difference between “evidence of a new particle” and “discovery of a new particle”?

Today’s exciting press release from Tevatron on the Higgs boson keeps its head cool and say that physicists saw a “hint” of the Higgs boson because the signal is barely above the two-sigma level. In an effort to explain the reasons for this, the press release says:

Physicists claim evidence of a new particle only if the probability that the data could be due to a statistical fluctuation is less than 1 in 740, or three sigmas. A discovery is claimed only if that probability is less than 1 in 3.5 million, or five sigmas.

I wonder: what's the difference between “claim[ing] evidence of a new particle” and “a discovery” (of said new particle)? Is this the common vocabulary for these levels (three and five sigmas) of signal?

The reason I ask is that I can understand the need for having different vocabulary to talk about discoveries with different levels of certitude, and I could see, say, the need for using “discovering a particle” and “confirming the discovery of a particle”, but “evidence” and “discovery” seem semantically very close to me.

Edit — to be clearer, the question is: how universal are the choices of terms (“evidence”, “discovery”) and associated incertitudes?

-
If that question about terminology is off-topic, please feel free to close it, of course! –  F'x Mar 7 '12 at 17:49
I'd say 'evidence' means a bump in the data that is debatable; 'discovery' is something that is extremely certain. Statistical meanings of 'debatable' and 'certain' most probably have to do with standard deviations as mentioned. Though if I were you I'd wait for @annav 's answer. –  Manishearth Mar 7 '12 at 17:53
@F'x: there was a time when this sort of thing might have been considered off topic, but the scope of the site has been getting broader over time and I think it's a fine (and important!) question. –  David Z Mar 7 '12 at 23:32

Evidence and Discovery are both meant in a probabilistic way. Although physics is commonly known as an "exact" science, you always have uncertainty in your measurements due to human errors and/or the measurements precision. Having said that, physicists in this area have ARBITRARILY chosen to accept a new discovery if and only if the probability of random noise in the data is extremely small (but never zero), and they have also chosen to accept that there is evidence of something if the probability of random noise is RELATIVELY small but is not sufficient to claim truthfullness with a strict arbitrary criterion of whats a discovery.

Edit: choice of terminology for evidence and discovery depends on the specific field or organization. One example outside physics is in econometrics where they ususally seek a significance of 0.1 or 0.05 to determine evidence in data. Maybe you can post your question in Cross Validated StackExchange as a Community wiki.

-
It's not entirely arbitrary, but is related to the embarrassingly many false "discoveries" that where made in the days of the particle zoo (mostly the 60s and 70s). –  dmckee Mar 8 '12 at 3:59
Arbitrary in the sense of why 5 sigma and not a little more or a little less to claim a discovery. 5 sigma is arbitrary but is a good rule of thumb. –  aortizmena Mar 8 '12 at 5:07

A discovery is evidence beyond doubt, in the sense that it is very unlikely to be revoked by further collection of data of the same kind.

Mere evidence is a conspiculous deviation from expectations, but the maount of data collected is too low, so that the event is not yet statistically significant with a high confidence level.

The message you quoted gives precise definitions for the thresholds used by the authors of the message: You need a deviation by 3 standard deviations to claim evidence, but one of 5 standard deviations to claim discovery. Baseed on the law of large numbers with its $N^{-1/2}$ dependence, one would conclude that it takes a factor of $(5/3)^2\approx 2.8$ more date to reach the discovery level, if the event is indeed real. (The associated probabilites are somewhat dubious, however, being based on the assumption that the deviations are normally distributed, which may or may not be the case.)

I don't know how universally the language is tied to the particular thesholds; it may be a CERN convention, but I doubt that it is a convention universally used in particle physics.

-
$5\sigma$ is pretty universal as far as I'm aware. –  David Z Mar 7 '12 at 21:14
Is there a widely known document defining the usage? I looked at the web site of the particle data group, but couldn't find out how to search there for such things. –  Arnold Neumaier Mar 7 '12 at 21:22
Not that I know of; it's just one of those things you pick up in the process of learning how experimental particle physics works. After all, this is a convention that basically emerged "organically," by consensus of the global HEP community, not something that was imposed by a standards organization or anything like that. It's possible that these terms might be explained in some textbooks, but I don't know of any specific examples. –  David Z Mar 7 '12 at 23:31