The rule of thumb I know for the error on a count for random sampling is that it goes as the square root of the count. For instance, if I observe a radioactive source for $1$ minute and measure $100$ decay events, a reasonable uncertainty on the count is $100\pm\sqrt{100} = 100 \pm 10$.

I have a different sampling problem, though. To put it simply, imagine I have a bag filled with a large number of balls which are each either red or blue. I'm interested in measuring the fraction of red balls in the bag. I draw an unbiased sample of $N_{\rm balls}$ balls and count the red ones, finding $N_{\rm red}$. My estimate for the red fraction $f_{\rm red}=N_{\rm red}/N_{\rm balls}$, but what uncertainty should I associate with this? I'm tempted to say $f_{\rm red}=\frac{N_{\rm red}}{N_{\rm balls}}\pm\frac{\sqrt{N_{\rm red}}}{N_{\rm balls}}$, but I find this unsatisfying. This is because it makes no distinction in the case where I draw $N_{\rm red}=0$; if this occurs with $N_{\rm balls}=1$, then my measurement is not especially constraining and should have a large uncertainty, but if I found $N_{\rm red}=0$ with $N_{\rm balls}=10^5$ I would have a high confidence that $f_{\rm red}\sim0$. In the particular experiment I have in mind, repeated draws are not an option.

Could someone point me to a correct handling of the statistics for such an experiment?

I appreciate that the physics content here is fairly minimal, so if the community prefers that this move to CV.SE or M.SE that's fair enough; I put it here because the context is a physical experiment and I would prefer answers in terms familiar to a physicist.


There is a problem here, in my opinion. You never can justify the guess $f=0$ rigorously it will always remain a hypothesis. If the bag contains an infinite number of balls and the fraction of red balls is $f\neq 0$, then a distribution of $N_{\mbox{red}}$ is the binomial distribution with the mean value $$ \overline{N_{\mbox{red}}} = N_{\mbox{balls}} f $$ and the variance $$ D = N_{\mbox{balls}} f (1-f). $$ In this case, $N_{\mbox{red}}/N_{\mbox{balls}}$ is near $f$ with hight probability only if $$ \overline{N_{\mbox{red}}} \gg \sqrt{D}. $$ This inequality is equivalent to the following one: $$ N_{\mbox{balls}} \gg \frac1{f}-1.\qquad(*) $$ So if $f \ll 1$, then $N_{\mbox{balls}}$ must be large enough to get a reliable estimation for $f$. If condition $(*)$ is not satisfied, then the probability of $N_{\mbox{red}} = 0$ is not small.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.