Error calculation for a relative difference with only one standard deviation (STD)

Question

I have a quantity that is calculated with two measurements, but I only have a tabulated standard deviation for one of the measurements. How would I calculate the error on the calculated quantity? The form is as follows: $\frac{a - b}{b}$. I have an std on $b$.

The problem is that I'm not sure how the standard deviation of $b$ is calculated. Both $a$ and $b$ are ozone measurements, but one is done with a ground-based instrument and the other is done with a satellite. The ground-based data has a cataloged std. These measurements are daily. The satellite data does not have an error estimate. I wonder if I should calculate my own estimate of the error on the satellite data or only use the cataloged std for the combination (relative difference) of $a$ and $b$? Because of seasonal variations in the data, I don't think I can just use all the data (or even one year) to estimate a standard deviation.

The ultimate goal is to put a $\sigma$ error bar on the relative difference of $a$ and $b$. Right now I'm just using the given std as an error bar and the results look okay, but I can't justify this approach. To give you an idea, my error bars are shown below. I find similar figures in this validation report, but the errors are not discussed in depth.

Any tips are greatly appreciated!

Answer 1

In general, scientific measurements reported without uncertainties are nearly worthless. The exception to that is where you have some good method of trying to estimate what the uncertainties were using the data itself. This is going to be a very data-specific problem and depends exactly on the characteristics of the thing being measured and how it was measured, so no specific answer can be essayed.

Suppose you have two sets of measurements of something but you know that the something does not vary and has a unique value. You could then estimate the uncertainty in the measurements of each of the two datasets by estimating their standard deviations from the individual datasets.

In your case I gather that the thing being measured does vary. In which case, the above method won't work. However, if you knew that the timescale of variation was greater than some timescale $\tau$ then you could get some sort of estimate of the intrinsic standard deviation of the measurements by splitting your dataset into chunks with time windows shorter than $\tau$. Each of these samples would give you an estimate (really an upper limit) of the standard deviation which could then be averaged to give you a final estimate.

You say that the measurements may vary "seasonally" but are taken every day. A more sophisticated approach would be to collect together the day-to-day differences (i.e. the difference between each consecutive pair of points). Under the assumption that these do not vary other than due to experimental errors then you can model the distribution of these differences to estimate the uncertainty in each measurement. This approach often has some intrinsic merit since it can reveal any non-Gaussian behaviour in the uncertainties.

As an aside - if someone presented me with the graph you have shown and told me that the error bars represent the (appropriately combined) errors in the two datasets, I would conclude either: the error bars have been massively overestimated; (ii) the data have been "fixed". The standard deviation of the plotted points should not be many times smaller than the estimated uncertainties in those points.

Answer 2

I agree that it is questionable to state a measurement without an estimate of the uncertainty. However, I do not believe that the data is almost useless. In order to proof my point I simulated the following:

• True temperature is a time series, where the random error is given by a normal distr. with mean value 0, and standard deviation 5. The details of the time series are not important and are therefore omitted here. The only important point is that we have some structure in the true temperature timeline, which is larger that the uncertainty of the measurement devices.
• The two measurements have a bias of 0.2 and -0.1, respectively,
• The two measurements have an uncertainty of 0.2, and 0.4, respectively. The random errors are normally distributed.
• The simulation uses 1000 data points.

This setup results in the following data:

As true temperature timeline has a structure which is considerably larger than the measurement uncertainties, we find correlations in the dataset. In this particular simulation we have

Correlations:
   cor(measTrue, meas1) = 0.958
   cor(measTrue, meas2) = 0.865
   cor(meas1, meas2)    = 0.829

Using this dataset we are able to calculate the difference between the two measured temperatures, and to fit a regression line. This yields the following

The first impression is that this is rather random and this is exactly what we want: The structure has disappeared. Thus, let's estimate the bias and the standard deviation from this dataset and compare it to the known bias and standard deviation of the difference. This yields:

* true bias: -0.3
* estimated bias: -0.29

* true SD: 0.447
* estimated SD: 0.446

The estimates are not bad at all.

Next, we could ask, how certain we are that the two sensors measure the same "same thing". For example, there might be a time lag between the two sensors, because they are at different positions. This assumption could be included in the simulation. What happens is that the estimated standard deviation increases. This is what we would expect, because the time lag induces uncertainty. Thus, you won't be able to infer the unstated uncertainty of the measurement device, but you might be able to estimate an upper bound for this uncertainty.

I expect that the presented simulation is not close to the dataset you have at hand. Therefore, you have to do the calculations yourself and certainly cross-check some assumptions you are willing to take. Nevertheless, it could be possible to estimate the uncertainty of the difference.