# Why don't experimental physics groups have statisticians in it?

Perhaps someone can clear up a bit a cognitive dissonance I am experiencing. Pollsters are under constant scrutiny of statisticians for even the most mundane of survey topics. With so much riding on the results of fundamental physics experiments, why don't we need statisticians to do the data analysis for us (or at least be looking over our shoulders)?

-
Isn't one answer that some groups DO have statisticians? – Greg P Nov 2 '11 at 19:20
In addition to the situations Greg notes, we also have some physicists who have spent more than the average amount of time on statistics, and our major code tools often know a bit of statistics themselves (i.e. if you use ROOT you don't need to know how to figure the Kolmogorov similarity of two distributions, just how to interpret the results). – dmckee Nov 3 '11 at 17:15

Because physicists learn the math and do it themselves. Why do you need a special expert class of people nowadays?

### EDIT: Deconstructing statistics

In response to comments that "statisiticians go through years of study", I would like to say why I think all this studying is counterproductive. The theory of statistics (when it isn't about statistical mechanics or pure mathematical measure theoretic things) is usually concerned with the inference problem--- what is the likelyhood of a parameter to be x when a measured quantity correlated with the parameter is measured to be $y_1,y_2,...,y_n$ in a sequence of trials.

The complete solution to this problem is given by Bayes's theorem: the probability that the underlying parameter has value x is given by the probability that this value x will produce the experimental results $y_1,y_2,..,y_n$ conditioned by the prior knowledge which gives you some distribution on x to begin with.

Because Bayes's theorem solves the problem of inference so simply and naturally, the field of statistics is almost entirely built on rejecting it. Most of the field is based on the idea that one should not do Bayesian inference for one cockamamie reason or another, usually based on some silly philosphy which rejects priors or rejects the notion of a fundamental a-priori notion of probability. Because of this, physicists never learn Baysian inference from a class, they have to rediscover it for themselves (I certainly did, and most other people who do inference do too).

This means that if you hire a statistician, they will most often find lousy workarounds for Baysian methods, which will be useless to the experimental physicist. The issue is deeply ingrained--- many famous topics in statistics, like the sufficient statistics or the t-test, are born of the quest for a non-Baysian inference This quest is misguided, and will waste the experimental physicist's time. Within statistics, however, anti-Baysianism is a useful motivation for new results, so the field is dominated by anti-Bayesians.

It is also true in Biology. There, the Baysian method is (with difficulty) replacing statistician's pet inference methodologies. This diatribe is based on experience from about a decade ago, and might be out of date.

-
But "the math" takes a statistician 5-6 years of Ph.D. study (plus 4 years of undergrad) to "learn". You need specialists because you can't put in 10 000 hours to become knowledgeable enough to employ some of the more sophisticated techniques required for some tasks -- and nowadays, indeed, more than ever! – Chris Ferrie Nov 2 '11 at 2:37
@Chris: any good physicist should be able to learn any necessry statistics in a month at the longest. If not, they are not particulary competent to do the experiments. It would be foolish to rely on someone else to do your statistics--- how would you be sure of any of your results? The reason statisticians go to school for so long is because PhD programs are not efficient in transmitting information, they are just there to provide a barrier to entry into a field. – Ron Maimon Nov 2 '11 at 3:54
@RonMaimon Since you find it so trivial, I'm sure the people studying open problems in statistics would love to talk to you. Here are 750 unanswered questions: stats.stackexchange.com/unanswered. I appreciate your extended answer but perhaps your point would be better taken if you didn't begin by being so glib. – Chris Ferrie Nov 2 '11 at 11:36
obligatory xkcd cartoon: xkcd.com/793 – EnergyNumbers Nov 2 '11 at 15:46
@Chris: I do not find statistics trivial, only the problem of inference. The statisticians are wrong on this problem, not the rest of the world. As for the xkcd crtoon, it is giving a brain-dead conservative politics, that implies that the physicist is wrong. Most academic fields consist of empty gibberish. Statistics is not empty, but the mathematics problems on the stats stackexchange are trivial, vague, or ill posed (I looked at a few). It is better to go to journals for open problems. – Ron Maimon Nov 2 '11 at 17:02

Maybe for the same reason that experimental physics groups do not have a theoretician as a group member.

One could think of experimental groups as ruled by "control freaks", they need as members experimetnalists who have mastered enough theory to set up the experiments and enough theory to interpret the results. Within this "theory" one could count statistics as the simpler one to master.

-
There's probably some truth to the "control freak" tendency, but you make it sound like a bad thing. A good group leader wants to raise future scientific leaders, not monkeys who are okay with a torque wrench but need a theorist and statistician on standby to get any science done. – wsc Nov 2 '11 at 21:31
Exactly, @Anna. – Luboš Motl Nov 3 '11 at 10:26
@wsc that is why I have the quotation marks around it. Of course strong leadership is necessary when one is working with tens, hundreds and thousands of scientists. Somebody once compared leading a group of physicists as "herding cats" . – anna v Nov 3 '11 at 11:54

In a number of areas in experimental physics you simply do not need a detailed statistical view on your data.

At the research institutes I have visited there were a great number of experimental and theoretical physicists but basically nobody that does anything that would need a deep understanding of statistical methods. While in some disciplines like particle physics the experiments and data often consists of counting events or occurrences this is not true for a lot of other disciplines.

As an example we are measuring resistance of metallic samples at low temperatures. We could measure the resistance 50 times per second, build a statistical model and so on, but we do not bother and it would be foolish. If the noise does not follow a gaussian distribution something is just wrong with our setup but not with the properties of our samples.

In many cases the possible systematic errors are much higher in magnitude and severity that it is a lot more efficient to minimize those than to reduce the statistical error by advanced mathematical methods.

-

One of the characteristics of physics research is the regular use of advanced methods and techniques from other fields, such as mathematics, computer science, probability theory and even biology. Physicists therefore often need to dive deep into complex topics in other fields. None of these topics is trivial and the level of understanding of physicists in these topics is as widely distributed as it is with complex 'hard core' physics topics.

-
I am constantly amazed at how much some experimentalist know (so much so that I feel odd categorizing them as such). They really are Jack's of all trades. But I probably could have said the same thing of my family doctor had I been alive 50 years ago. Now all he/she does is refer me to specialists. It's not that he/she is incapable of doing these things. It's better to think of it as he/she is capable of interfacing with the specialists. Wouldn't the experimental physics community benefit (or eventually require) this same dynamic? – Chris Ferrie Nov 3 '11 at 11:18

Because physicists does not need most of statistics -- they only do hypothesis testing, with systems following simple laws and with directly measured, uncluttered data.

-

If your experimental result is shown to have a statistical significance of 3 sigmas only by employing the brightest statisticians using the most cutting-edge modern techniques, while the more pedestrian statistical techniques that a typical physicist knows only produce 2 sigmas, then you have trouble convincing the wider community of the real importance of your experiment.

While arguing over the detailed statistic techniques can affect the statistical significance level by some amount, improving your experimental methods potentially will boost your result much more dramatically. It's good for physicists to focus on the latter rather than the former.

P.S. The most convincing experiments are those that don't require statistics at all. When an astronomer claims he has discovered a star, he typically doesn't quote any statistical significance level. When the significance is above 20, or even 100 sigmas, the word "statistics" simply disappears for good.

-