Percent error calculations dilemma

I have a set of experimental results for calculating g:

9.82

9.52

10.77

10.39

9.75

9.79

10.13

10.56

10.26

9.84

10.07

9.58

These were taken using a pendulum experiment. My dilemma is that, to calculate the percent error should I:

1. Use the average value of 'g' from the above; or
2. Calculate the % error of EACH trial, then average that one out taking into account negative quantities (eg -2.4% would be 2.4%)

If I do the first, it won't be as accurate because negative values can offset the final result. Percent errors should be treated as inequalities, yes?

The second one I believe is much more accurate although tedious. Using the first method, my % error was 2.24% which does not seem right. Using the second method it is 3.33%. This is more plausible.

I've been doing it like method one the whole year and I've never been marked down for it per se, and so has everyone in my class. My instructors didn't pick up on this.

Is my reasoning invalid?

The usual method for estimating errors takes the RMS (root mean squared) value - which is actually the same as the standard deviation of the sample.

The first method you describe is "take the answer I should have gotten and see how different my answer was". That works when the answer is known, but it is in general a terrible way to do science (where you don't know the answer).

Often, you want to know the error in a measurement to see whether a particular hypothesis is supported by your data. A rather infamous example was the Grasso experiment that seemed to indicate (to within the error of the measurement) that they were observing particles traveling faster than the speed of light. The subsequent discovery of a systematic error debunked the whole thing... but it was an interesting application of the concept of error estimation in the formulation of hypotheses.

Your second proposed method takes the absolute mean error: this is OK-ish, but it gives less weight to outliers.

The standard method computes the standard deviation - which any calculator or spreadsheet will help you do. For comparison, your data gives the following results:

method       result
vs known     2.3%
abs error    3.2%
rms error    3.9%


It is often a good idea to create a plot of your data to see if there are any obvious outliers. If you have software that allows you to plot a normal plot, that would be ideal; but even a linear plot of sorted values can be instructive:

In this case there are no obvious outliers. If you want to give your audience a sense of how broadly distributed the data were that made up the measurements, you would state your result as

10.0 ± 0.4 $m/s^2$ (mean ± standard deviation, n=12)

As Rob Jeffries pointed out, if instead you want to give an indication of your confidence that the actual result lies within a certain range, you need to quote the standard error - which is the standard deviation divided by the number of samples, or

$$S.E. = \sqrt{\frac{\sum{(x_i-\bar x)^2}}{n(n-1)}}$$

Since the standard error is much smaller than the standard deviation of the data (because making lots of measurements increases your confidence in the result), you can actually quote the result to an additional significant figure:

$$mean = 10.04 \pm 0.11 m/s^2$$

With thanks to Rob whose comments inspired me to improve the answer.

When I look at your data, I see that your best estimate of the value of $g$ is $10.04 \pm 0.11$ m/s$^2$. This would be your quotable result.

The error bar here is the standard error in the mean; defined as the corrected sample standard deviation divided by the square root of the number of measurements. i.e. $$\Delta \bar{x} = \sqrt{\frac{1}{N}} \sqrt{\frac{1}{N-1} \sum_{i=1}^{N} (x_{i} -\bar{x})^2} =\frac{0.39}{\sqrt{12}}$$

You say you wish to calculate the percentage error, but you don't say the percentage error in what?

If it is in your best estimate of $g$, then that is $(0.11/10.04)\times 100 = 1.1$ per cent.

If it is an estimate of the percentage error in each reading then, providing we reasonably assume that $g$ is constant and that there is no systematic drift in your measurements (i.e. they are scattered around the mean without any trend in time or any other parameter), this is given approximately$^{1}$ by the corrected standard deviation of the population divided by the mean value $\times 100$.

i.e. Percentage error in a single reading $= (0.39/10.04)*100 = 3.9$ per cent.

The above assumes you have well-behaved data. And you should always have a look at it to check for outliers. Unfortunately, this can be a bit of a circular process. The presence of outliers leads to a bigger standard deviation and in a set of 12 points you might expect 4 to be more than one standard deviation from the mean in a normal distribution.

So there are alternative strategies; more robust estimators of the deviation/uncertainty. My favourite in the datsets I look at, which often suffer from non-Gaussian, extended tails, is the median absolute deviation which is robust to outliers, and can be multiplied by 1.48 to recover a Gaussian-like 68% confidence uncertainty.

Edit: I've been queried on why I quote $\pm 0.11$ as the error on the mean. Well, that is the "standard error in the mean". It is what you would use to say whether your value of $g$ agreed with an accepted value or someone else's value. For example, your result looks a bit high. It is $0.23\pm 0.11$ m/s$^2$ higher than the gravity where I live.

A good discussion of sample vs population standard deviation and standard error in the mean can be found on the relevant wikipedia pages

$^1$ Note that the "corrected sample standard deviation" (the square root with $N-1$ in the denominator above) is an unbiased estimate of the variance, but may be a biased estimator of the true standard deviation (in this case true error of the measurements), especially if $N$ is small. See the discussion here, which suggests that using $N-1.5$ might be more accurate if you know the sample follows a normal distribution.