14
$\begingroup$

Suppose we have a probability per time $\lambda$ that something (e.g. nuclear decay, random walk takes a step, etc.) happens. It is a known result that the probability that $n$ events happen in a time interval of length $T$, is given by the Poisson distribution $$P(n) = \frac{e^{-\lambda T} (\lambda T)^n}{n!} \, .$$ How do we prove this?

$\endgroup$
  • $\begingroup$ Ha - I saw this question and I thought "weird, I would have expected Daniel to know the answer to this". Then I scrolled down... $\endgroup$ – Floris Dec 4 '17 at 23:13
  • $\begingroup$ @Floris well it took me a while to get help on the integral... $\endgroup$ – DanielSank Dec 4 '17 at 23:25
  • $\begingroup$ I wrote an alternative that doesn't explicitly need integrals $\endgroup$ – Floris Dec 4 '17 at 23:54
  • $\begingroup$ @Floris yes, and I like the explicit relation to the binomial distribution, but I'm not sure that approach directly answers this question. Perhaps you can argue that many little time slices are like many low-probably attempts... $\endgroup$ – DanielSank Dec 4 '17 at 23:56
  • $\begingroup$ Yes - that's exactly what I argue. In the limit of infinitely many time slices with the product $pN$ remaining constant, you are effectively making the transition from discrete to continuous - without ever appearing to integrate. $\endgroup$ – Floris Dec 4 '17 at 23:57
12
$\begingroup$

Probability distribution of time until next event

First we calculate the probability density that a time $t$ passes without any event happening. Divide $t$ into $N$ small intervals each of length $dt = t/N$. Defining $\lambda$ as the probability per time that the event occurs, the probability that no event occurs within any one short time interval is approximately $(1 - \lambda dt)$. Therefore, the probability that no event happens in any of the intervals, but then does happen within an interval of length $dt$ right at the end of all the intervals is $$\prod_{i=1}^N \left( 1 - \lambda dt \right) \lambda dt = \left( 1 - \frac{\lambda t}{N} \right)^N \lambda dt \stackrel{N\rightarrow \infty}{=} \lambda dt \exp \left( - \lambda t \right) \, .$$ In other words, given a starting time $0$, the probability density that no event has happened after a time $t$, but then happens right at $t$ is $\lambda \exp(-\lambda t)$.

Probability of multiple events

Now we ask the probability that we get $n$ events in the time interval $T$. Suppose the first event happens at $t_1$, the second even happens at $t_2$, etc. We therefore have a series of intervals $$\{[0, t_1], [t_1, t_2], \ldots [t_n, T] \} $$ with events happening at the end of each interval. The probability that our events occur in this way is $$P(t_1, t_2, \ldots t_n) = \lambda \exp(-\lambda t_1) \lambda \exp(-\lambda (t_2 - t_1)) \cdots \lambda \exp(-\lambda(T - t_n)) = \lambda^n \exp(-\lambda T) \, .$$ Of course, any arrangement of $\{t_1, t_2, \ldots t_n \}$ such that $t_1 < t_2 < \ldots < t_n$ counts as an $n$-event arrangement, so we have to add up the probabilities of all these possible arrangements, i.e. the probability of $n$ events is \begin{align} P(n \text{ events}) &= \int_0^T dt_1 \int_{t_1}^T dt_2 \cdots \int_{t_{n-1}}^T dt_n P(t_1, t_2, \ldots t_n) \\ &= \lambda^n \exp(-\lambda T) \int_0^T dt_1 \int_{t_1}^T dt_2 \cdots \int_{t_{n-1}}^T dt_n \, . \end{align} The multiple integral is the volume of a right simplex and has value $T^n/n!$, so the final result is $$P(n\text{ events}) = \frac{(\lambda T)^n \exp(-\lambda T)}{n!} $$ which is the Poisson distribution with mean $\lambda T$.

Related

$\endgroup$
  • $\begingroup$ In your first derivation should $\lambda =1$ produce a probability density of 1 everywhere. If the probability density of the event happening at $t=t'$, and not before, is $\lambda e^{-\lambda t'} dt$ then $lambda = 1 \implies PDF(t)=e^{-t} dt$ then for any $t'$ we should get $ p(t') = \int_{0}^{t'} PDF(t) = -e^{-t'} + 1$ not $1$. [mod edit to fix MathJax - DZ] $\endgroup$ – D. W. Dec 5 '17 at 0:38
  • $\begingroup$ @D.W. I don't understand this comment. $\lambda$ has dimensions of 1/time, so it can't have value 1. Could you clarify? $\endgroup$ – DanielSank Dec 5 '17 at 19:14
  • $\begingroup$ Forgive me, I'm not criticising your answer, or suggesting you're wrong, I'm just trying to understand your derivation for myself. Isn't $\lambda$ the probability per unit time that something happens? What if something is guaranteed to happen? Wouldn't that correspond to a value of $\lambda = 1$ (the probability per unit time that the event happens is 1) $\endgroup$ – D. W. Dec 5 '17 at 19:17
  • $\begingroup$ @D.W. I know you're not criticizing. No problem. In fact, the case you're describing where the event always happens even for infinitessimally small times would correspond to $\lambda \to \infty$. You can plug that into the various equations at any point and see how the results turn out. Does that help? $\endgroup$ – DanielSank Dec 5 '17 at 19:36
  • $\begingroup$ Yes, thank you. I'm still a little confused as to how an event having a probability one of happening at all times corresponds to a probability per unit time that approaches infinity, but that's for me to ponder. $\endgroup$ – D. W. Dec 5 '17 at 19:43
6
$\begingroup$

The Poisson distribution describes the probability of a certain number ($n$) of unlikely events ($p\ll 1$) happening given $N$ opportunities.

This is like doing a very unfair coin toss $N$ times, with the probability $p$ of the coin turning up heads. The number of heads would follow the binomial distribution:

$$P(n|p,N) = ~^{N}C_n~p^n (1-p)^{N-n} =\frac{N!}{(N-n)!~n!} p^n (1-p)^{N-n}$$

Now it remains to prove that when $N\rightarrow \infty$ and $p\rightarrow 0$ while $Np\rightarrow \lambda T$, that the above converges to the known result. In essence, I argue that when you make the number of opportunities tend to infinity, you go from a discrete to a continuous approach; but as long as you are careful with your infinities, the result should still be valid.

First, we find an approximation for $(1-p)^{N-n}$. Taking the log, we get

$$\log\left((1-p)^{N-n}\right) = (N-n)\log(1-p)\approx (N-n)\cdot (-p)$$

Since $N\gg n$, we obtain $(1-p)^{N-n}\approx e^{-Np}$

Next, we approximate the $~^N C_n$ term using Stirling's approximation that $\log N! \approx N\log N - N$ and noting that $n\ll N$

Then

$$\begin{align} \log\left(\frac{N!}{(N-n)!}\right) &= N\log N - N - (N-n)\log(N-n) + (N-n) \\ &=N\log N - (N-n)\log(N-n) - n\\ &= N \log N -(N-n)\left(\log(N)+\log\left(1-\frac{n}{N}\right)\right) - n\\ &\approx N\log N -(N-n)\left(\log(N)-\frac{n}{N}\right) - n\\ &\approx n\log N + n -n \log n - n\\ &=n\log(N-n)\end{align}$$

It follows that $\frac{N!}{(N-n)! n!} \approx \frac{N^n}{n!}$

Finally, we note that $pN = \lambda T$, and we get

$$P(n|N,p) = \frac{N^n p^n e^{-Np}}{n!}$$

this easily rearranges to

$$P(n) = \frac{(\lambda T)^n e^{-\lambda T}}{n!}$$

Which is the result we set out to prove.

I made use of this article to remind me of some of the steps in this.

$\endgroup$
  • $\begingroup$ Minor nitpick: it's actually Stirling's approximation, not Sterling's $\endgroup$ – WetSavannaAnimal Dec 5 '17 at 8:06
  • 3
    $\begingroup$ Actually, don't worry if you think that's bad, I spelt Max Planck's name for years the same as the piece of wood or the title of the short film by Eric Sykes. And I know German pretty well (or well enough for much of my communications for my work). I was 50 before I was corrected. That's not the worst of it. I had spent several month long or more visits to Max Planck Institute before I caught on. $\endgroup$ – WetSavannaAnimal Dec 5 '17 at 8:12
5
$\begingroup$

Let $A_{n,t}$ be the event: exactly $n$ events happened over a time interval $t$. Then

$$P(A_{n,t+\Delta t}) = P( A_{n,t} \cap A_{0,\Delta t }) + P(A_{n-1,t} \cap A_{1,\Delta t }) \, .$$

Using independence of occurrence and the definition $\lambda = \lim_{\Delta t\to 0} P(A_{1,\Delta t})/\Delta t$, taking the limit, and defining $P_n(t) \equiv P( A_{n,t})$, we get the recurrence $$P'_n(t)= \begin{cases} -\lambda ( P_n(t) - P_{n-1}(t) ) & n>0\\ -\lambda P_n(t) & n=0 \end{cases}$$

with the initial conditions $$P_n(0)=\begin{cases}0 & n>0\\1 & n=0 \, .\end{cases}$$

The solution is given by the Poisson distribution:

$$P_n(t)=\frac{(\lambda t)^n \exp(-\lambda t)}{n!} \, .$$

$\endgroup$
  • 1
    $\begingroup$ I've learnt two new ways to prove this today, aside from Floris's answer, which was the one I already knew :). Most elegant! $\endgroup$ – WetSavannaAnimal Dec 5 '17 at 7:53

protected by Qmechanic Dec 5 '17 at 4:01

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.