How do you find the uncertainty of a weighted average?

Question

The following is taken from a practice GRE question:

Two experimental techniques determine the mass of an object to be $11\pm 1\, \mathrm{kg}$ and $10\pm 2\, \mathrm{kg}$. These two measurements can be combined to give a weighted average. What is the uncertainty of the weighted average?

What's the correct procedure to find the uncertainty of the average?

I know what the correct answer is (because of the answer key), but I do not know how to obtain this answer.

Answer 1

I agree with @Ron Maimon that these ETS questions are problematic. But this is (i think) the reasoning they go with. Unlike @Mike's assumption you should not take the normal average, but as stated in the question the weighted average. A weighted average assigns to each measurement $x_i$ a weight $w_i$ and the average is then

$$\frac{\sum_iw_ix_i}{\sum_i w_i}$$

Now the question is what weights should one take? A reasonable ansatz is to weigh the measurements with better precision more than the ones with lower precision. There are a million ways to do this, but out of those one could give the following weights:

$$w_i = \frac{1}{(\Delta x_i)^2},$$ which corresponds to the inverse of the variance.

So plugging this in, we'll have

$$c = \frac{1\cdot a+\frac{1}{4}\cdot b}{1+\frac{1}{4}}= \frac{4a+b}{5}$$

Thus,

$$\Delta c = \sqrt{\left(\frac{\partial c}{\partial a}\Delta a\right)^2+\left(\frac{\partial c}{\partial b}\Delta b\right)^2}$$

$$\Delta c = \sqrt{\left(\frac{4}{5}1\right)^2+\left(\frac{1}{5}2\right)^2}=\sqrt{\frac{16}{25}+\frac{4}{25}}=\sqrt{\frac{20}{25}}=\sqrt{\frac{4}{5}}=\frac{2}{\sqrt5}$$

which is the answer given in the answer key.

Why $w_i=1/\sigma_i^2$

The truth is, that this choice is not completely arbitrary. It is the value for the mean that maximizes the likelihood (the Maximum Likelihood estimator).

$$P(\{x_i\})=\prod f(x_i|\mu,\sigma_i)=\prod\frac{1}{\sqrt{2\pi\sigma_i}}\exp\left(-\frac{1}{2}\frac{\left(x_i-\mu\right)^2}{\sigma_i^2}\right)$$. This expression maximizes, when the exponent is maximal, i.e. the first derivative wrt $\mu$ should vanish:

$$\frac{\partial}{\partial\mu}\sum_i\left(-\frac{1}{2}\frac{\left(x_i-\mu\right)^2}{\sigma_i^2}\right) = \sum_i\frac{\left(x_i-\mu\right)}{\sigma_i^2} = 0 $$

Thus, $$\mu = \frac{\sum_i x_i/\sigma_i^2}{\sum_i 1/\sigma_i^2} = \frac{\sum_iw_ix_i}{\sum_i w_i}$$ with $w_i = 1/\sigma_i^2$

Answer 2

Since other questions as well as Google point to this question, I would like to extend the already existing answer from luksen by a motivation from the stochastic for using the error propagation equation.

Let's assume we have $n$ random measurements $X_i$, typically denoted by $E\{X_i\} \pm \sigma_i$ (or $x_i \pm \Delta x_i$), whereas $E\{\cdot\}$ and $\sigma_i$ denote the expected value and the standard deviation, respectively. According to the question, we are interested in the weighted average of these measurements, calculated by $Y = \frac{\sum_i w_i X_i}{\sum_i w_i}$ ($=c$). Thanks to the linearity of the expected value, it is rather easy to get $$E\{Y\} = E\left\{\frac{\sum_i w_i X_i}{\sum_i w_i} \right\} = \frac{\sum_i w_i E\{ X_i\}}{\sum_i w_i}.$$ For the variance $\sigma^2$, we need a few more lines but no hacky tricks. $$\begin{align} \sigma^2 &= E\left\{(Y-E\{Y\})^2\right\} = E\left\{\left(\frac{\sum_i w_i X_i}{\sum_i w_i} - E\left\{\frac{\sum_i w_i X_i}{\sum_i w_i} \right\} \right)^2\right\} \\ &= \frac{1}{(\sum_i w_i)^2} E\left\{ \left(\sum_i w_i X_i\right)^2 - 2 \left(\sum_i w_i X_i\right) \left(\sum_j w_j E\{X_j\}\right) + \left( \sum_i w_i E\{X_i\} \right)^2 \right\} \\ &= \frac{1}{(\sum_i w_i)^2} E\left\{ \sum_{i,j} w_i X_i w_j X_j - 2 \sum_{i,j} w_i X_i w_j E\{X_j\} + \sum_{i,j} w_i E\{X_i\} w_j E\{X_j\} \right\} \\ &= \frac{1}{(\sum_i w_i)^2} \sum_{i,j} w_i w_j \left( E\{X_i X_j\} - 2 \cdot E\{X_i\} E\{X_j\} + E\{X_i\} E\{X_j\} \right) \\ &= \frac{1}{(\sum_i w_i)^2} \sum_{i,j} w_i w_j \left( E\{X_i X_j\} - E\{X_i\} E\{X_j\} \right) \\ &= \frac{1}{(\sum_i w_i)^2} \sum_{i,j} w_i w_j \cdot Cov(X_i, X_j) \end{align}$$ The covariance holds $Cov(X_i, X_i) = \sigma_i^2$ and, as long as the original measurements $X_i$ and $X_j$ are independent, $Cov(X_i, X_j) = 0$ otherwise. This leads to the answer of the original question, the variance (squared standard deviation) of the average $Y$ of $n$ measurements $X_i$ with random uncertainty $\sigma_i$: $$\sigma^2 = \frac{1}{(\sum_i w_i)^2} \sum_i w_i^2 \sigma_i^2,$$ which yields $$\Delta c = \sigma = \frac{1}{\sum_i w_i} \sqrt{\sum_i (w_i \sigma_i)^2}.$$ That's exactly what the error propagation equation yields after inserting the derivations of $c$. Why is it exact while the error propagation equation is an approximation? The error propagation equation approximates by a first-order Tayler expansion. Since the average is a linear function, it is exact and not only approximated, here.

Additional information: For the unweighted average ($w_i = 1~ \forall i$), we get $$ E\{Y\} = \frac{1}{n} \sum_i E\{ X_i\} \quad\quad\quad \sigma^2 = \frac{1}{n^2} \sum_i \sigma_i^2$$ If all original samples $X_i$ have the same variance $\tilde\sigma^2$, this leads also to the well know variance of the average: $$\sigma^2 = \frac{\tilde\sigma^2}{n}.$$

Answer 3

It would be highly unreasonable to obtain the uncertainty of the average of the measurements to be greater than the uncertainty of any one measurement (√5/2 > 1). After all, what's the point in taking averages if it just makes your readings more uncertain?

I believe the ETS people used the argument that the harmonic sum of the individual variances should give the reciprocal of the average's variance i.e. 1/v = 1/v1 + 1/v2, as detailed here: http://en.wikipedia.org/wiki/Weighted_arithmetic_mean#Dealing_with_variance