Correction factor for standard deviation

Question

I have conducted an experiment, from which I obtained a finite amount of values for the gravitational acceleration. The amount of values ranges from 6 up to ~150 values.

I have the unbiased sample variance and the (approximated) standard deviation defined as follows:

$$ \begin{align*} s^2 &= \frac{1}{n-1} \sum\limits_{i=1}^n \left(x_i - \bar{x}\right)^2\\ s &= \sqrt{s^2} \end{align*} $$ ($n$ is the total number of values, $x_i$ is the i-th value, $\bar{x}$ is the arithmetic mean,
note that I forgot the factor 1/(n-1) in a previous revision of this post)

I have been given the following formula for the mean error for few measured values: $$ \Delta x = \frac{t}{\sqrt{n}}s $$ Where $t$ is to be looked up in the next table:

+-------------------------+-------------------+
| Number of single values | correction factor |
+-------------------------+-------------------+
| 5                       | 1.15              |
| 6                       | 1.11              |
| 8                       | 1.08              |
| 10                      | 1.06              |
| ... (cut)               | ... (cut)         |
| more                    | 1.00              |
+-------------------------+-------------------+

Question: Which correction factor should I take for 7 single values?
Shall I round up to the next available correction factor?

My teacher is currently unavailable, I therefore hope this question is objectively answerable with the information given.

Answer 1

You are trying to estimate the mean of population that your data come from, and you are assuming a normal distribution for the data.

However, you do not know the true variance of the population, and you need to use the data to estimate the variance as well as the mean. Therefore, you don't get quite as much "information" out of your experiment as you would have hoped: you have to use up some of the "discovery power" to estimate another parameter of the distribution (i.e. variance/standard deviation) before you can begin to do the "real work" of estimating the mean. Thus, the estimated means come from a distribution that is a little bit broader than the normal - the Fisher t-Distribution of $n-1$ degrees of freedom. As your sample size $n\to\infty$, the difference between the Fisher and Normal distributions gets smaller and smaller.

You could, therefore, look up your value from tables of the Fisher t distribution. However, with your given values, you could do a linear interpolation and get a pretty good result. That is, $7$ is half way between $6$ and $8$, so the mean of $1.11$ and $1.08$ i.e. $1.095$ would be a good value to use. Notice that the decrease between $8$ and $10$ is less than decrease betweem $6$ and $8$; this means your curve is "concave up" (its rate of decrease decreasing), so that $1.095$ will be a slight overestimate of the true value. This is good. It means you will not be assuming any more accuracy than your experiment can give you.

Answer 2

Just one more look at this... given the numbers, you could fit a curve through the data. If you decided to include one additional point on either side of the missing value (4 points total) you could fit an exact cubic, and the interpolated value would be 1.100; if instead you did an (inexact) quadratic fit through the four points you would get 1.093. A linear interpolation through the two adjacent points gives 1.095 (it would not be a good idea to fit a straight line through the four points as there is obvious curvature). Finally, you could do the interpolation by computing the curvature from the four points, then applying that curvature as a correction to the linear interpolation between the two middle points.

Here is what these "fits" to the data would actually look like:

where I plot $n$ along the horizontal, and the quadratic/cubic fit to the four correction values given along the vertical.

Of course since the correction values were only given to 2 sf it is hard to justify getting too wrapped around the axle about the difference - the cubic fit is clearly a poor representation of the underlying data (see how the slope changes sign...).

With that said, I think that WetSavannaAnimal's approach (linear interpolation, accepting that the value is probably a little bit conservative) is the right one.

Or you can go back to theory - but it's tricky and while it may be "statistically correct" to fret about these differences, it is practically almost never worth your while (except to get a good grade which gets you into the right university which gets you a good job which ...)

Answer 3

I agree with @WetSavannaAnimal as far as interpolating to get the correction factor goes.

Alternatively you could go and find out what the correction factor for 7 values actually is.

However, the formulae you originally quoted for the unbiased sample variance and the sample standard deviation were incorrect.

The formula for unbiased variance is $$s^2 = \frac{1}{n-1} \sum_{i}^{n} (x_i - \bar{x})^2 $$

When you take the square root of this you get a biased estimator of $s$, the standard deviation. This all discussed here, where a formula is given for a correction factor to $s$ that is appropriate if you know that your data are normally distributed, which usually, you don't.

I am puzzled by your correction factors. Were you told that the data followed something other than a normal distribution? There are approximations that can be used to calculate the corrections for a normal distribution. In the form that you define it, the value for $n=7$ is 1.0423 and the correction factor for other $n$ values do not match the ones in your table.

?