How to compare several experimental measurements and their uncertainties?

Question

I have a basic question about how to make the comparison of different measurements. For example, suppose I have measured the focal length of a convergent lens using three different methods, $A$ and $B$ and $C$, and I have obtained the following values and uncertainties:

$$\left({f^\prime}_{A}\pm\Delta{f^\prime}_{A}\right)=(0.3865\pm0.0003)\ m$$

$$\left({f^\prime}_{B}\pm\Delta{f^\prime}_{B}\right)=(0.3861\pm0.0005)\ m$$

$$\left({f^\prime}_{C}\pm\Delta{f^\prime}_{C}\right)=(0.3853\pm0.0002)\ m$$

I see that the first two results are consistent since the intervals formed by their values and uncertainties share common points, but the third does not coincide with them.

How is the usual way (in scientific articles, laboratory reports, etc.) of expressing these aspects? Maybe something like this?

$$[{f^\prime}_{A}-\Delta{f^\prime}_{A},{f^\prime}_{A}+\Delta{f^\prime}_{A}]\cap[{f^\prime}_{B}-\Delta{f^\prime}_{B},{f^\prime}_{B}+\Delta{f^\prime}_{B}]\neq \emptyset$$

Answer 1

If you perform multiple (pair-wise) hypothesis tests, the so called $\alpha$ risk (type I error) increases. Therefore, better method is to use an ANOVA or a $\chi^2$ goodness of fit test to compare all results at once. Only if we find a significant result, we should compare the pairs to find out, which "data point" is responsible.

Answer 2

One way to express whether a pair of measurements are consistent with each other is to look at the difference of the quantities with its associated uncertainty. So, for example, you could define $\delta_{AB} \equiv f_A - f_B = 0.0004$ m. Assuming that the errors in $f_A$ and $f_B$ are Gaussian and uncorrelated and correspond to the standard deviations of the underlying distributions, then we can also define the uncertainty in $\delta_{AB}$ using the usual sort of rules. In this case, it works out to be $0.00058$ m.

You would then report $\delta_{AB} = 0.0004 ± 0.0006$ m. Since $\delta_{AB} = 0$ is within this range, there is no evidence of any inconsistency between these two measurements.

Note that by this logic (and under these assumptions), the evidence of inconsistency between measurements $B$ and $C$ is not terribly strong; $\delta_{BC} = 0$ is within the $2\sigma$ range of the distribution for $\delta_{BC}$.