I am not sure how to determine the statistical error in a measurement. As an example, let's assume that we measure a quantity $X$ $n$-times and get $x_1,\cdots,x_n$. The corrected sample standard deviation is given by $$s = \sqrt{\frac{1}{n-1} \cdot \sum_{k=1}^n (x_k-\overline{x})^2} $$ Now here comes my question: Some people now tell me that this is the error on $x$. (result: $\overline{x} \pm s$) On the other hand, I have also been told that you have to use $s/\sqrt{n}$ for the error. And as a third option, one said, you have to extend that by a factor t. My question is now, which option is correct and why.
-
$\begingroup$ Correct is s if you state that that is the standard deviation, dividing it by $\sqrt{n}$ is not standard, but since the standard variation only says that the true value ist in the 67% limit, you can extend it with a factor t to 90% or 99% $\endgroup$– trulaAug 20, 2022 at 13:37
-
2$\begingroup$ Would Cross Validated be a better home for this question? $\endgroup$– Qmechanic ♦Aug 20, 2022 at 13:54
-
$\begingroup$ Does this help? $\endgroup$– J.G.Aug 20, 2022 at 16:32
-
1$\begingroup$ I suggest that you read John Taylors book. $\endgroup$– RussellHAug 20, 2022 at 18:17
3 Answers
The standard uncertainty, the one that you would report as the uncertainty of the measurement, is given by $s$.
$s/\sqrt{N}$ would be the uncertainty in the mean of a sample of $N$ measurements, which is different from the uncertainty of the individual measurements. This would not be what you report as the uncertainty of the measurement, but instead this would be what you report as the “standard error of the mean”.
As far as why that is correct, that is simply how the BIPM (the international bureau of weights and measures, the organization that is in charge of the SI) have defined things. It is the convention that the scientific community as a whole follows so that people can communicate clearly.
-
$\begingroup$ The BIPM is a late-comer here: these conventions are much older, with roots in the Central Limit Theorem. And until fairly recently, this sort of error analysis was more common in other fields: sociology, biology, epidemiology... $\endgroup$ Aug 20, 2022 at 20:35
-
1$\begingroup$ The reason I cite the BIPM is not historical, so the timing is unimportant. The important part is that the BIPM is considered authoritative today and that the SI standards are generally required by journal editors today $\endgroup$– DaleAug 20, 2022 at 20:39
The answer to your question is that it depends on what you are trying to discuss/present. However, its important that we clearly state what we are presenting:
- The sample standard deviation is $s= ...$,
- The standard deviation of the average value is $s/\sqrt{N} = ...$ -- this is often called standard error, or standard error of the mean.
- The 95% confidence interval for the average value is $\bar x \pm 1.96 \, s/\sqrt{N} = ...$.
Discussion:
Let's suppose we have $N$ measurements $\{x_1, x_2, \ldots, x_N\}$. We can write the $k^{th}$ measurement results as $$x_k = \mu + \epsilon_k$$ to emphasise that it consists of two components:
- the true value $\mu$, and
- a random error $\epsilon_k$ from the normal distribution with mean value 0 and standard deviation $\sigma$.
To estimate the true value $\mu$ and the standard deviation $\sigma$ using the dataset $\{x_1, x_2, \ldots, x_N\}$, we commonly use:
- the sample average $\bar x = \frac{1}{N}\sum_{i=1}^N x_i$ as estimator for $\mu$, and
- the sample standard deviation $s = Sd[x] = \sqrt{ \frac{1}{N-1}\sum_{i=1}^N (x_i-\bar x)^2}$ to estimate $\sigma$. This is the first option of the upper list.
In order to arrive at the second and third option of the upper list we have to change our focus. While the sample standard deviation $s$ is the uncertainty associated with a single measurement $x_k$, we are often most interested in the uncertainty of the average value $\bar x$. Our intuition tells us that the uncertainty of the average value $Sd[\bar x]$ is smaller than the uncertainty of the single measurement $s=Sd[x]$, because the random errors partially cancel each another. The "proof" of our intuition is given by so called central limit theorem: It tells us that (under certain conditions) the average value of a dataset is normally distributed with mean value $\mu$, and standard deviation $\sigma/\sqrt{N}$. Therefore, the uncertainty of the average value is $Sd[\bar x] = Sd[x]/\sqrt{N} = \sigma/\sqrt{N}$ -- note the bar over the $x$. This is the second option from our list.
Finally, we could express our confidence in the obtained average value by stating a confidence interval. By using the factor $t=1.96$ (often this is approximated by 2) we obtain the 95% confident interval of the average value, which is $\bar x \pm 1.96 \cdot Sd[\bar x] = \bar x \pm 1.96 \cdot s/\sqrt{N}$. This is the final option of our list. Also note, that the factor $t$ differs from $1.96$, if the sample size is "small" and we need use a $t$-distribution.
In the upper paragraphs we discussed all three presentations you mentioned. Therefore, we obtain my initial statement: Clearly state what you are presenting, then all three options are fine.
-
$\begingroup$ You are correct. I was confused by the wording. The distributions I have been working with for the last decade of my life have long tails. Taking averages makes very little sense in those scenarios. With exception of thermal noise almost none of the error calculus that I had to do in my life was for a standard textbook distributions. In most cases there was not even a known expression for the shape of the distribution. $\endgroup$ Oct 9, 2022 at 13:16
-
$\begingroup$ Yeah, I am too old and too jaded, I guess. I use simple statistics all the time to get an order of magnitude for the error and then I automatically assume that I am wrong by tens of percent for that error, at least. I also agree whole heartedly that the state of teaching with regards to statistics is completely insufficient. I apologize for the initial remark. I truly misunderstood what you were trying to say. $\endgroup$ Oct 9, 2022 at 21:01
John Taylor's book, Introduction to Error Analysis should answer all your questions in this regard.
I think that $\frac{s}{\sqrt{n}}$ is not right. There are two forms for standard deviation (SD) of the average of measured values:$$s=\sqrt{\frac{1}{n}\sum_{k=1}^{n}(x_{k}^{2}-\overline{x}^{2})} $$$$ s=\sqrt{\frac{1}{n-1}\sum_{k=1}^{n}(x_{k}^{2}-\overline{x}^{2})}$$
The first one (population SD) is used for a large number of measurements. The second one (sample SD) is corrected by Bessel to recognize that the SD of 1 measurement is undefined.
So for a small set of measured values use the Sample SD. For a large set use the Population SD.
In either case the expression:$\overline{x}\pm s$ means that 67% of the measured values lie within 1 SD of the average.
The expression:$x_{meas}\pm \delta x$ means that the measured value has an uncertainty ($\delta x$) of 1SD. You must indicate if the uncertainty is 2SD or 3SD. Example, $x_{meas}\pm \delta x$ 2SD. This is equivalent to t=2 or t=3.
So you report the error or uncertainty based on what your data set is and on what you are trying to convey.
Standard deviation applies to data that is randomly correlated. The average does not reveal truth.
Read Taylor.
-
1$\begingroup$ The Bessel correction uses $n-1$ and not $n+1$. $\endgroup$– SemoiOct 9, 2022 at 8:37
-