# Correct method to determine uncertainty in success rates

Let's say I have a situation where I'm counting success and failure events giving me two values of interest

$$s = \frac{a}{b} = \frac{\text{# of successful outcomes}}{\text{total # of trials}}$$ $$f = \frac{b-a}{b} = \frac{\text{# of failed outcomes}}{\text{total # of trials}}=1-s$$

Intuitively I would think that the uncertainties of both variables needs to be the same, $\delta s = \delta f$. Where $\delta a,\delta b$ are determined by Poisson statistics, where the uncertainty in a count is simply the square root of that count.

However, if I use the method of error propagation through a function

$$\delta s = \sqrt{\left(\frac{1}{b}\delta a\right)^2 + \left(\frac{a}{b^2}\delta b\right)^2}$$

I get a different uncertainty from doing the same but for $\delta f$. Therefore, it seems as though something is missing. Is it as simple as some function of $a$ and $b$ that needs to be added into the function for $\delta s$ giving

$$\delta s = \sqrt{\left(\frac{1}{b}\delta a\right)^2 + \left(\frac{a}{b^2}\delta b\right)^2+C(a,b)}$$

where $C$ will make $\delta s = \delta f$ for any $a,b$.

Or is it that this method of error propagation simply cannot be applied here?

Unfortunately, talking to some of the experimentalists around here, I get different answers, some say that I should simply take the smaller of the two uncertainties.

• This isn't really a physics question.
– Omry
Dec 5 '16 at 16:11
• Why do you get a different uncertainty for $\delta f$? That seems to be the problem here, the derivatives of $a/b$ and $1-a/b$ are certainly the same up to sign... Dec 5 '16 at 16:37
• If I use the propagation of uncertainties through a function as shown above for $\delta s$ I get for example, $83/90 \pm\delta s$ but if I do the same for $\delta f$ I get $7/90\pm\delta f$, where $\delta f \neq\delta s$. However, if I find $\delta f$ by using the fact that $f = 1-s$ I get that $\delta f = \delta s$. So, depending on my approach, I get different answers. P.S. should I move this to math Stack Exchange?
– John
Dec 5 '16 at 18:11
• I really think that physics majors should be required to take a course of real statistics, but many of the courses offered direct a lot of time and effort on analyzing the kind of data seen by (the more numerous) social and biological scientists to the detriment of treating systems of interest to physicists; and physics major are often near the top of the required course hours as it is. Which is why, while I agree with @Omry in principle, I think the question belong on Physics anyway: because physicists teach the skillset that leads to this confusion to physics majors. Dec 5 '16 at 18:31

The naive rules usually given to first year students have two built in assumptions:

• That all the errors are uncorrelated.

• That each error is small enough that higher order terms may be neglected.

These assumptions are actually mentioned briefly in most treatment but subsequently left unstated and get washed out of students minds by the complexity of what follows.

A full treatment in the case that either assumption is violated adds much more complexity, so the issue is generally left unraised even where it ought to be discussed,

In this case, even though $$\delta a$$ and $$\delta b$$ may be uncorrelated, we know that $$\delta(b - a)$$ is correlated with $$\delta b$$, so trying to find $$\delta f$$ by applying the usual rules to the expression $$(b-a)/b$$ is certainly an error, while applying them to $$1 - (b/a)$$ is OK as long as $$\delta a$$ and $$\delta b$$ are uncorrelated.

• I found a very short, concise treatment of uncorrelated, completely correlated, and partially correlated values at phenix.bnl.gov/WWW/publish/elke/EIC/Files-for-Wiki/…. For me it looks to be a good starting point for further research into the subject.
– John
Dec 5 '16 at 18:38
• In this case however you don't need the more complicated cases: $a$ and $b$ are assumed to be uncorrelated, so you just need to take your derivatives with respect to $a$ and $b$, leading to the same result whichever form you use for $f$ as long as you take the derivatives right and plug them into en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplification . It was always a puzzle for me why this isn't taught first and the special cases derived from it. Dec 5 '16 at 22:13