I am working on this lab that involved gathering data from two different sources. It involved gathering reaction times from a device and from a web application which was put into our data sets.

It is asking about anticipated events and if they agreed with the experimental uncertainty. After Googling these terms, I still found the explanations very unclear. Can someone explain how I could go about answering this?


You'll likely need to give a few more details to get exactly the help you need but I'd like to say following because what you are being asked is actually the grounding of all science: you are being asked to test whether the data you witness falsify an underlying theory (i.e. whatever it is that foretells your anticipated events).

Many, if not most, modern philosophers of science agree that one of the main things that defines science is its falsifiability - a concept introduced by Karl Popper who came up with the epistemology "Falsificationalism".

So what you are being asked to do in your lab might seem a little boring, but you are striking at and glimpsing the very foundations of all science in its ideas.

True science must foretell propositions about the world which can, even if only in priciple, be shown true or false by an experiment. The statistical notion of hypothesis testing is nothing less than the quantification of Popper's revolutionary idea, and as such it is statistics which is the Queen of Sciences (rather than simply mathematics, as Carl Friedrich Gauss said). When aplied properly, there is no more powerful way known to humanity to find out truth with.

So, in short, you are being asked, "If such and such a theory is true, how likely would you be to see the observed results?". If the likelihood is highish, then there is no reason to doubt the theory. If the likelihood is low, but not too so, you may simply be seeing a statistical fluctuation, thus you probably won't consider a theory falsified - although you may want to repeat the experiment and pool results for confirmation. If it is outlandishly low, you consider the results inconsistent with the theory in question and the theory is falsified by these results.

As far as uncertainties go, let's suppose a theory makes some foretelling given data $a_1, a_2, \cdot$. This foretold result will be some function $f(\{a_j\})$ of the data. Now, of course, you are never going to see exact agreement between foretelling and observation. But there are uncertainties in your data, and statistical fluctuations in your data will beget a corresponding "cloud" of possible values in $f$. So we think of the $a_j\in A_j$ wandering about in little open balls $A_j$ around their "supposed" values, and these balls represent the uncertainty in your observations - the precision of your instruments, for example. The "cloud" of values that you will plausibly see if your theory is true is the image set $f(A_1,\,A_2,\,A_3\,\cdots)$. If the uncertainties in $a_j$ are small, then we can calculate the "cloud" of plausible values as follows. The total variation in$f$ given variations $\delta a_j$ in your data are approximately:

$$\delta f \approx \sum_j \frac{\partial f}{\partial a_j} \, \delta_{a_j}$$

so, if the data $a_j$ have means $\mu_j$ and variances $\sigma_j^2$, then often a good statistical model for the variation in $f$ is that $f$ is a normally distributed random variable with mean and variance given by:

$$\mu = f(a_1, a_2, \, \cdots)$$ $$\sigma^2 = \sum_j \left(\frac{\partial f}{\partial a_j}\right)^2 \, \sigma_j^2$$

and then you take your observed value of $f_o$ and test whether its deviation fro $\mu$ is likely, given your data. In other words, you test the null hypothesis that the observed $f_o$ is actually foretold by $f(a_1, a_2, \, \cdots)$.

If you give some more details of your data, I can help you apply these ideas.

In the light of Michael Brown's Comments:

Some of the things @MichaelBrown is talking about in his comments can be gotten a feel for from the Stanford Encyclopedia of Philosophy a most excellent resource, particularly under the pages:

  1. Bayesian Epistemology;

  2. Interpretations of Probability; and

  3. Chance versus Randomness

You'll quickly see that rigorous underpinnings of propability and statistics are a work in progress, but it is important to keep a good grip on the fact that statistical methods that work are here and very powerful, or, a good grip on Michael's pithy summary:

"The ultimate goal is to confront our ideas with experiment"

Nonetheless, given the importance of statistics and its foundational relationship with the scientific method, the rigorous underpinnings of probability must be an exciting area to work in, and absolutely vital research to further. An amusing question that gets put on this site is "why can't quantum mechanics use classical probability" - as though probability would be an easier thing to grasp! At least one can ask Nature for the answer in QM by doing an experiment!

Lastly: There is also quite a beautiful readable exposition of the subjectivist (Baysean) / frequentist "dichotomy" of interpretations in probability and statistics to be found in the opening section of E. T. Jaynes, "Information Theory and Statistical Mechanics", Phys. Rev. 106, number 4, pp 620-630, 1965. Jaynes also shows how in Physics it is impossible to choose one over the other - both are fundamental and indispensable.

  • $\begingroup$ So the 'sigma' I keep reading about in articles on high energy physics, is actually the variance in f? $\endgroup$ – dj_mummy Sep 24 '13 at 4:18
  • 1
    $\begingroup$ @dj_mummy Standard deviation. $\sigma^2$ is the variance. $\endgroup$ – Michael Brown Sep 24 '13 at 4:19
  • $\begingroup$ It is perhaps worth mentioning, without entering into a long and hairy discussion, the two schools of statistical hypothesis testing: the frequentist and Bayesian schools. Roughly speaking the frequentists imagine repeating an experiment an infinite number of times (which you obviously can't actually do) and relate statistics to frequencies within this imagined ensemble of experiments. (Again roughly speaking) the Bayesians refer to probabilities as "degrees of belief" or "certainty of a proposition" and relate statistics to these mental attitudes rather than imagined frequencies. $\endgroup$ – Michael Brown Sep 24 '13 at 4:28
  • $\begingroup$ Both schools invoke subjective assumptions in their methods: the frequentists in assuming properties of an ensemble of imaginary experiments, and the Bayesian in explicit "prior probabilities." These lead to somewhat different outlooks, methods, and in the past fighting between the two groups, though it has been shown that the best methods of both groups tend to agree with each other - some are even identical apart from the interpretation. But the ultimate goal is the same for everyone though: to confront our ideas with experiment. $\endgroup$ – Michael Brown Sep 24 '13 at 4:30
  • $\begingroup$ @MichaelBrown Don't know whether you're aware of Jaynes's treatment of this stuff, but I just recalled there is a wonderful exposition of it by him - see the very end of my answer where I have added the reference. $\endgroup$ – WetSavannaAnimal Oct 7 '13 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.