5  Poisson distribution

\[ \require{physics} \require{braket} \]

\[ \newcommand{\dl}[1]{{\hspace{#1mu}\mathrm d}} \newcommand{\me}{{\mathrm e}} \]

\[ \newcommand{\Exp}{\operatorname{E}} \newcommand{\Var}{\operatorname{Var}} \newcommand{\Mode}{\operatorname{mode}} \]

\[ \newcommand{\pdfbinom}{{\tt binom}} \newcommand{\pdfbeta}{{\tt beta}} \newcommand{\pdfpois}{{\tt poisson}} \newcommand{\pdfgamma}{{\tt gamma}} \newcommand{\pdfnormal}{{\tt norm}} \newcommand{\pdfexp}{{\tt expon}} \]

\[ \newcommand{\distbinom}{\operatorname{B}} \newcommand{\distbeta}{\operatorname{Beta}} \newcommand{\distgamma}{\operatorname{Gamma}} \newcommand{\distexp}{\operatorname{Exp}} \newcommand{\distpois}{\operatorname{Poisson}} \newcommand{\distnormal}{\operatorname{\mathcal N}} \]

Poisson distribution is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time or space if these events occur with a known constant mean rate and independently of the time since the last event.

Definition 5.1 Let \(\mathcal Y=\set{0,1,2,\ldots}\). The uncertain quantity \(Y\in\mathcal Y\) has a Poisson distribution with mean \(\theta\) (denoted by \(Y\sim\distpois(\theta)\)) if \[ \Pr\qty(Y=y\mid \theta)=\pdfpois(y,\theta)=\dfrac{\theta^y\me^{-\theta}}{y!}\propto\theta^y/y!. \]

Theorem 5.1 If \(Y\sim \distpois(\theta)\), then

• \(\Exp\qty[Y|\theta]=\theta\),
• \(\Var\qty[Y|\theta]=\theta\),
• \(\Mode\qty[Y]=\lfloor\theta\rfloor\).

Proof

\[ \begin{split} \Exp\qty[Y|\theta]&=\sum_{y=0} y\,\pdfpois\qty(y,\theta)=\sum_{y=0} y\,\frac{\theta^y\me^{-\theta}}{y!}=\me^{-\theta}\sum_{y=1}\frac{y}{y!}\theta^{y-1}\theta\\ &=\theta\me^{-\theta}\sum_{y-1=0}\frac{\theta^{y-1}}{(y-1)!}=\theta\me^{-\theta}\me^{\theta}=\theta,\\ \Exp\qty[Y^2|\theta]&=\sum_{y=0} y^2\,\pdfpois\qty(y,\theta)=\theta\me^{-\theta}\sum_{y-1=0}\frac{\theta^{y-1}}{(y-1)!}y\\ &=\theta\me^{-\theta}\left(\sum_{y-1=0}\frac{\theta^{y-1}}{(y-1)!}(y-1)+\sum_{y-1=0}\frac{\theta^{y-1}}{(y-1)!}\right)\\ &=\theta\me^{-\theta}\left(\theta\me^{\theta}+\me^{\theta}\right)=\theta^2+\theta,\\ \Var\qty[Y|\theta]&=\Exp\qty[Y^2|\theta]-\Exp\qty[Y|\theta]^2=\theta^2+\theta-\theta^2=\theta. \end{split} \] To find the \(\Mode\), we check the relation between two consecutive terms.

\[ \dfrac{\pdfpois(y,\theta)}{\pdfpois(y-1, \theta)}=\frac{\theta^{y}\me^{-\theta}/y!}{\theta^{y-1}\me^{-\theta}/(y-1)!}=\frac{\theta}{y}. \] \(\pdfpois(y,\theta)\) is increasing when \(\theta/y\geq1\) (which means that \(y\leq \theta\)), and decreasing when \(\theta/y\leq 1\) (which means that \(y\geq \theta\)). Since \(y\) has to be an integer, we have

\[ \Mode\qty[Y\mid\theta]=\lfloor\theta\rfloor. \]

Theorem 5.2 (Moment generating function) Let \(Y\sim\distpois(\theta)\). Then \[ M(t)=\Exp\qty[\me^{tY}]=\exp(\theta(\me^t-1))=\me^{\theta(\me^t-1)}. \]

Proof

\[ \begin{split} \Exp\qty[\me^{tY}]&=\sum_{y=0}^{\infty}\me^{ty}\pdfpois(y,\theta)=\sum_{y=0}^{\infty}\me^{ty}\frac{\theta^y\me^{-\theta}}{y!}\\ &=\sum_{y=0}^{\infty}\frac{\qty(\theta\me^t)^y\me^{-\theta}}{y!}=\me^{-\theta}\sum_{y=0}^{\infty}\frac{\qty(\theta\me^t)^y}{y!}=\me^{-\theta}\me^{\theta\me^t}\\ &=\me^{\theta(\me^t-1)}. \end{split} \]

5.1 Relations between Poisson distribution and binomial distribution

Consider the number of events happen during a fixed length period. This is a Poisson process. Assume that the expectation of the distribution is \(\theta\). This means that we could expect \(\theta\) events happening during the time. The distribution of the process is \(\pdfpois(y, \theta)\).

Poisson distribution is the limit of binomial distribution

\[ \begin{split} \lim_{n\rightarrow\infty}\pdfbinom(y,n,\theta/n)&= \lim_{n\rightarrow\infty}\binom{n}{y}\qty(\frac{\theta}{n})^y\qty(1-\frac{\theta}{n})^{n-y}\\ &=\lim_{n\rightarrow\infty}\frac{n!}{y!(n-k)!}\frac{\theta^y}{n^y}\qty(1-\frac{\theta}{n})^n\qty(1-\frac{\theta}{n})^{-y}\\ &=\frac{\theta^y}{y!}\lim_{n\rightarrow\infty}\mqty[\frac{n(n-1)(n-2)\ldots(n-y+1)}{n^y}\qty(1-\frac{\theta}{n})^{-y}]\lim_{n\rightarrow\infty}\qty(1-\frac{\theta}{n})^n\\ &=\frac{\theta^y}{y!}\me^{-\theta}\\ &=\pdfpois(y,\theta). \end{split} \]