After I watched "Particle Fever"--the movie about Large Hadron Collider (LHC) and the successful identification of the Higgs boson--I became a bit concerned with that team's handling of various negative PR incidents. Further, with the amount of money spent and the pressure to produce results, also realized that we may not sometime soon have a way to reproduce any of their results at any other facility with other research teams.

With this in mind, how much scientific confidence can we put into things like the mass of the Higgs? Not that a "Higgs-like" particle was found, but the actual calculated mass of the particle known as the "Higgs boson"?

This is a question about the scientific method and reproducibility of experimental results. Do we have any similar experiments where we confirm a similar theory without being able to reproduce those results?

UPDATE: What is the confidence level of the LHC-calculated mass of the Higgs? How was this confidence level determined? What are the implications of it being wrong? How long will it take before we know if it is wrong?

Thank you all for your comments. I do believe this is an important, specific and scientific discussion that can have specific, factual answers.

UPDATE 2: I guess I'm not the only one asking these questions, this is very interesting:


“The current data is not precise enough to determine exactly what the particle is. It could be a number of other known particles,” Dr Frandsen said.

A related question might be, based on the Frandsen et al paper, what if it is not the Higgs at all?

UPDATE 4/9/15: Came across this re: the reversal of the BICEP2 "discovery" due to having a 2nd team and 2nd set of instruments via the Planck telescope. Without Planck, the BICEP2 team might still be claiming their CMB discovery... perhaps for for a long time to come, potentially leading cosmology research (time and dollars) down the wrong cosmic inflation "rabbit hole." This seems to be an example supporting the importance and relevance to the question I raised here regarding LHC and acceptance of its "discoveries" without a 2nd team using a 2nd set of instruments (ie a 2nd beam and collider): http://physicsworld.com/cws/article/news/2015/feb/03/galactic-dust-sounds-death-knell-for-bicep2-gravitational-wave-claim

  • 3
    $\begingroup$ Please keep your sense of cost into perspective: the total cost of the LHC is comparable to a cost of an airplane carrier , of which the US has several. A boat can sink and the investment is lost for the future. The LHC data will be studied for decades. (there are still papers coming out of LEP data) In addition LHC is built with the financial contribution of many countries. $\endgroup$ – anna v Mar 23 '15 at 6:29
  • $\begingroup$ The use of the word "trust" in this scientific context is not appropriate. The scientific method demands validations, not acts of faith. $\endgroup$ – Andrew Palfreyman Mar 24 '15 at 19:21
  • 1
    $\begingroup$ @anna v: What is the cost of getting it wrong and assuming it is correct for decades, sending thousands of scientists down the wrong rabbit hole? $\endgroup$ – Brad Cooper - Purpose Nation Mar 28 '15 at 21:07
  • 1
    $\begingroup$ Accumulating data are not the wrong rabbit hole. If people had not been experimenting we would still at best be in the sixteenth century of technology. BTW, LHC is reproducing a great number of measurements from LEP and the Tevatron, using them as a gauge for exploring further. $\endgroup$ – anna v Mar 29 '15 at 3:26
  • 2
    $\begingroup$ This is definitely a better question after the edit, although as to whether it is enough better to reopen, we'll see what people think. $\endgroup$ – David Z Mar 29 '15 at 15:33

Most of the reproduction of results in particle physics comes from two sources:

  • Competing experiments running nearly simultaneously.

    In this case both ATLAS and CMS got comparable results. Now, they are both using the beam from the LHC, so how do we know the beam is properly understood? Because while they were commissioning those machines they reproduced dozens (literally multiples of twelve) results that don't actually require the full capacity of the LHC. Notably they re-discovered the top quark and re-measured its mass, lifetime and branching ratios.

    Moreover, CERN has a long and very successful history in characterizing beams. They know how to do that stuff.

  • Commissioning of the next-generation machine.

    This, of course, doesn't do us any good just now, but it is coming. Indeed, the LHC is the machine answering the same question about the Tevatron (a decade ago you could, after all, have asked something similar about CDF and D0 discovering the top quark).

As an aside, don't underestimate the extent to which ATLAS and CMS are different machines, different cultures and differ in almost all the particulars. Nor the degree to which each of these organizations is looking over the other's shoulders: yes they put on a collegial show when announcing their results together but the competition is fierce and they do take each other's reports apart in search of things to criticize.

Second aside: a lot of biomedical research has had a problem in recent years with a lack of reproductions at all. And it is a issue when studies are expensive (which is true in particle physics) and money is tight (when isn't it?), so how do we come out ahead in this matter? As far as I can tell it is a feature of building machines that are flexible, and developing a deep pool of talent when it comes to turning the data-set around to examine it from another angle.

  • 1
    $\begingroup$ Thank you. If there were something truly nefarious going on behind the scenes, it seems that the rediscovering of the top quark and "accurately" re-measuring its mass could be just be a way to re-enforce that they "know how to do that stuff" and put higher confidence by extension on the accuracy of the Higgs measurements? If they re-measured the top quark and got it "wrong", more questions would arise on the validity of their other measurements. $\endgroup$ – Brad Cooper - Purpose Nation Mar 28 '15 at 21:03
  • 1
    $\begingroup$ I'm not sure you understand how big collaborations work. There is internal cross-checking, and grad-students and post-docs have a strong need to stand out (in a positive way!) if they are to get a good job. Being the guy or gal who found a major error is a great way to start developing a reputation as a first rate scientist. There is also competition between different member institutions, so no PI can be confident that of bulling the point. You are positing a conspiracy with 10,000+ participants many of whom are going to be booted from academia in the long run (some are bitter about it). $\endgroup$ – dmckee --- ex-moderator kitten Mar 28 '15 at 21:11
  • 2
    $\begingroup$ big collaborations, all working from the same beam and same source data, yes? How many of those 10,000 really have hands-on visual calibration & inspection of the inner workings? Can also imagine a grad-student saying, "hey wait a minute...I know we have just spent $15B, but we have to shut everything down and fix XYZ and start over because..." Unlikely and probably not a great career ahead for that grad student. Collaborations may not have been as big or diverse, but space shuttle disasters give us some indications of where big mistakes can be missed. $\endgroup$ – Brad Cooper - Purpose Nation Mar 29 '15 at 15:45

The first difficulty in making a measurement of this kind is, of course, to build the facility. However once you solve all the engineering issues and put your machines in place, the experiment flow is pretty straightforward: you run and you get the data, in principle without any human intervention. Up to here you just let Nature do its work in a pretty objective way, but now comes the real difficulty: how to interpret all those signals from the detector components in order to get a single number and an uncertainty estimation?

The key here is to be open to the world. This means that you (a physicist at whatever institute) can access not only all the detectors drawings and detailed descriptions, but also the raw data collected by them. You can develop your analysis algorithms to reconstruct the events and actually do a measure in a totally independent way.

When measuring something one of the most critical aspects is the instrument calibration. But even here, if you do not thrust the work done at CERN, your can do your own looking at particle with known mass (like the $Z$, $W^{\pm}$ bosons and many others).

There has been this Atlas challenge where laymen were asked to provide a discrimination algorithm to distinguish between different (preprocessed) events. Going from there to the very raw data is not trivial, but could be done by an independent team. An independent measure done from scratch at Atlas or CMS would as valuable as a measure done at a new facility.

And regarding the colliding beams, you do not really have to know anything. Indeed accelerating protons, which are composite particles, you do not know which fundamental particles actually collided (gluons, quarks,...) and even less you know their collision energy. So yes, they are the same beams at Atlas and CMS, but that does not matter at all, because there are no informations from the beam side that are fed into the measure.


The question is "how much scientific confidence can we put into things like the mass of the Higgs". Well, what is the level of certainty? According to CERN it is around 7 sigma. In simple terms at 7 sigma, both the CMS and ATLAS teams are reporting that there’s only a 0.0000000001% chance that they haven’t found a Higgs-like particle.

  • 2
    $\begingroup$ That assumes they're estimating their error correctly. $\endgroup$ – Mark Mar 23 '15 at 7:58
  • 2
    $\begingroup$ That is what peer review is for. It also assumes others can check the raw data. OTOH I don't think it fair to criticize the fact that there is only one machine that can perform the experiment. Do you also criticize the Hubble telescope for being the only one of its kind as well? $\endgroup$ – user56903 Mar 23 '15 at 9:00
  • $\begingroup$ So according to the team doing the measurements, they think they are very accurate? Well, that's a relief. We have Kepler and soon will have James Webb. $\endgroup$ – Brad Cooper - Purpose Nation Mar 28 '15 at 20:57
  • $\begingroup$ Also, a "higgs-like" particle could still not be the Higgs, correct? What is the level certainty on the MASS and how can they possibly know that? $\endgroup$ – Brad Cooper - Purpose Nation Mar 28 '15 at 21:16

Do we have any similar experiments where we confirm a theory without being able to reproduce those results?

Not today, but once we know how it works we can repeat it on a smaller and cheaper scale.

The first electronic calculators required an entire floor, consumed staggering amounts of power and cost a budget-busting amount of money. Today, my phone can run a Colossus or IBM 7000 emulator without interrupting my Youtube video.


Not the answer you're looking for? Browse other questions tagged or ask your own question.