1
$\begingroup$

I think there is a simple answer out there but I have hit a silly mental block.

Say I have two values, each with their own error (precision):

$845.9 \pm 0.3, 845.6 \pm 0.3$

Now, I understand that by applying one of the errors to the value suggests that the values are in fact possibly the same. However, I have been trying to deduce this by calculating the difference in the values and propagating the error to that value ("if the calculated difference value $\pm$ error spans $0$, the two values can be justified to be the same" is my reasoning).

However, I have two ways which I think I can calculate this:

  1. $(845.9 - 845.6 ) \pm \left|\sqrt{(0.3^2) + (0.3^2)}\right|$
    Here the result is $0.3 \pm 0.42$, which spans $0$.

  2. looking at the extreme pairs of the original values:
    $845.9+0.3, 845.6-0.3$ (worst case, largest difference)
    $845.9-0.3, 845.6+0.3$ (best case, smallest difference)
    which results in two differences between the extreme values: $0.9$ and $-0.3$.
    Here the average value is again $0.3$, but the error would be $\pm 0.6$. This value for the difference also spans zero, but over a greater domain (larger estimate of the error).

What is the difference between the two methods of calculating the differences? The most intuitive to me is option 2), however I suspect that this method is not correct. Any insights would be greatly appreciated!

$\endgroup$
2
  • $\begingroup$ Is the second measurement 845.6 or 849.6? $\endgroup$
    – Paul
    Jan 28, 2020 at 20:07
  • $\begingroup$ Thank you, it is 845.6, I just edited my question to reflect this correction. $\endgroup$
    – D. Hallatt
    Jan 29, 2020 at 17:36

2 Answers 2

3
$\begingroup$

The error bars are different because they measure different confidence intervals.

On the original measurements, a symmetric error bar characterized with a single number can typically be thought of as the standard deviation of a Gaussian distribution with the mean located at the value of the measurement. If one were to re-measure the value, there is a roughly 68% chance that the measurement would fall within these bounds. Sometimes, instead of the standard deviation, the error bars refer to the 95% confidence interval, which is just the width of two standard deviations of the same Gaussian distribution (so there's a 95% chance that a follow-up measurement will be within the error bars in that case).

Any reputable study will tell you at least which of these it is using (ideally, it should tell you a lot more than that), so refer to the context of these measurements when in doubt. In any case, it doesn't really matter which one we pick for what we're about to discuss, so we'll just assume that the error bars refer to one standard deviation.

When you have measurements with Gaussian errors like this, the distribution of the difference of the measurements will also be a Gaussian distribution. The value you get from the normal propagation of uncertainty formula is the standard deviation of the Gaussian distribution representing the difference of the two measurements. In other words, the propagation of uncertainty formula gives you the width such, if you re-measured both measurements and took their difference, the result would be within the error bars 68% of the time.

Examining the second method, it should be clear that what you're actually doing is calculating the sum of the standard deviations of the measurements, as opposed to the square-root of the sum of the squares of them.

Note that usually, the two measurements are statistically independent - the value of the first one doesn't depend on the second (again, check the context of these measurements to make sure this is true; often in real life it isn't, and there's some component of correlated error that influences both measurements the same way). This means that having both measurements at the extremes of their error bars simultaneously is a rather unlikely event; after all, if one event is remeasured to be one error bar away from its mean, there's a possibility that the other event is remeasured as, for example, the same value as before or anywhere else in that range. In fact, this kind of simultaneous extreme measurement actually falls outside the range of events that happen 68% of the time, which means that any error bars which include it will necessarily be wider than 1 standard deviation.

So what kind of confidence interval are you actually measuring, then? We can calculate this explicitly. Let $\sigma_a$ and $\sigma_b$ be the standard deviations of the two measurements, and let $\sigma_{a-b}$ be the standard deviation of the difference of the two measurements. We know from error propagation that $\sigma_{a-b}=\sqrt{\sigma_a^2+\sigma_b^2}$, and the interval you're interested in has width $\sigma_a+\sigma_b$. So your interval has a width of $\frac{\sigma_a+\sigma_b}{\sqrt{\sigma_a^2+\sigma_b^2}}$ standard deviations (this is commonly called the Z-score). Plug this into any of the many Z-score to probability converters found on the internet and you have the percentage of events that your confidence interval covers.

For the special case of $\sigma_a=\sigma_b$, as it is here, the Z-score is $\frac{2\sigma_a}{\sqrt{2\sigma_a^2}}=\sqrt{2}$. This means that your second confidence interval is wide enough that roughly 84% of remeasurements will be within it. This is slightly less than the 95% confidence interval calculated from two standard deviations, and greater than the 68% confidence interval corresponding to one standard deviation, which makes sense, because your interval corresponds to $\sqrt{2}$ standard deviations.

$\endgroup$
9
  • $\begingroup$ Thank you. This is very concise and well thought out. With regards to my exact data, the error bars I quoted were obtained from the manufacturer of the recording instrument as the apparent "precision" of the instrument. Unfortunately, as you hinted at, I have been unable to identify what this metric really is (2 sigma, 1 sigma who knows). I think the thing I took away from your post was the probability of hitting both extreme values. Considering this in method #2 inflates the % required to hit that event (which is more than whatever original % the error is referring to, 68% or 95%). $\endgroup$
    – D. Hallatt
    Jan 29, 2020 at 18:26
  • $\begingroup$ I do have one question though, I am confused as why you referred to the error as being systematic. If my understanding is correct, wouldn't a known systematic error (known because I quoted it as being a known +/- 0.3), just be able to be negated by subtraction without the need for stats. If I'm not mistaken I think these errors would be imprecisions instead which are stochastic and can't be calibrated out, hence the need for probabilities and stats. $\endgroup$
    – D. Hallatt
    Jan 29, 2020 at 18:36
  • $\begingroup$ @D.Hallatt Reread the answer, the word "systematic" doesn't appear anywhere. The error bars are symmetric, but the errors are not systematic. If you're doing this for a professional purpose, I would definitely contact the manufacturer and ask how specifically they're defining precision. $\endgroup$ Jan 29, 2020 at 18:49
  • $\begingroup$ My mistake, I must be seeing things from my thesis I am currently writing. I am sourcing my precision error from the manufacturer b/c I am unable to generate my own from repeat measurements. I have contacted them as you suggested. Thanks again for everything! $\endgroup$
    – D. Hallatt
    Jan 30, 2020 at 15:58
  • $\begingroup$ @D.Hallatt Out of curiosity, why are you unable to generate your own precision from repeat measurements? Is it because you no longer have access to the device, or is there something about the measurements themselves or the device itself that prevents this? $\endgroup$ Jan 30, 2020 at 16:09
0
$\begingroup$

You can use a standard approach to this problem, and get a standard solution. But it might not fit your problem.

If your particular problem is worth doing right, look at the details of how you get your measurement and try to make your best guess at what kind of errors you get.

Is your measurement guaranteed to be within the [-0.3,0.3] region around the right value, but just as likely to be anywhere inside that region? Are some of the causes of error guaranteed to result in underestimates? Overestimates?

The natural assumption is that there are many small causes for error that are all independent, and the total error is the sum of the little ones. But that might not be so. If what you actually get is some random errors divided by other random errors, you could get some wild results.

If you look at all the causes of error you can think of and they are the sum of a lot of small errors, then you're set. Use the standard methods.

Otherwise you might try using simulation. Make a computer program that handles errors the way you think they happen. run it hundreds or thousands of times, and make a graph. Look at the graph and decide how you would represent a graph like that as [-x, y]. You might for example want to have 5% of the area of the graph outside your confidence interval.

An even better possibility -- somehow you got a lot of data about the errors by comparing true values against measured values, that's how you got the [-0.3, 0.3] error estimate in the first place. Use THAT data in your computer program. Pick random pairs of known errors. Then you don't have to estimate the error sources and likely get it wrong.

There's the problem that there could be rare LARGE errors. Your known data won't have many of those, because they're rare. But your estimated errors will only have them if you noticed the possibility, and you might likely have a wrong guess just how rare they are. There's no good way to deal with that, unless you can find a way to estimate them from very large data sampling.

There's the problem that your computer program's random number generator is not completely random. This will probably not get in your way.

There's the problem that roundoff error will eventually build up. As you decrease the uncertainty of your results with bigger sample sizes, eventually you start increasing it with roundoff error. This will probably not get in your way. These are just things to notice that can cause problems occasionally, particularly as the scale gets great big.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.