6 Beta distribution
\[ \require{physics} \require{braket} \]
\[ \newcommand{\dl}[1]{{\hspace{#1mu}\mathrm d}} \newcommand{\me}{{\mathrm e}} \]
\[ \newcommand{\Exp}{\operatorname{E}} \newcommand{\Var}{\operatorname{Var}} \newcommand{\Mode}{\operatorname{mode}} \]
\[ \newcommand{\pdfbinom}{{\tt binom}} \newcommand{\pdfbeta}{{\tt beta}} \newcommand{\pdfpois}{{\tt poisson}} \newcommand{\pdfgamma}{{\tt gamma}} \newcommand{\pdfnormal}{{\tt norm}} \newcommand{\pdfexp}{{\tt expon}} \]
\[ \newcommand{\distbinom}{\operatorname{B}} \newcommand{\distbeta}{\operatorname{Beta}} \newcommand{\distgamma}{\operatorname{Gamma}} \newcommand{\distexp}{\operatorname{Exp}} \newcommand{\distpois}{\operatorname{Poisson}} \newcommand{\distnormal}{\operatorname{\mathcal N}} \]
Definition 6.1 (Beta distribution) \(\theta\sim\distbeta(\alpha, \beta)\) means that \[ \Pr(\theta)=\pdfbeta(\alpha, \beta;\theta)=c\cdot\theta^{\alpha-1}(1-\theta)^{\beta-1} \] where \(c=\dfrac{1}{B(\alpha,\beta)}=\dfrac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}\).
\(\pdfbeta(1,1)\) is actually uniform.
6.1 Expected values
Theorem 6.1 Let \(\theta\sim\distbeta(\alpha,\beta)\). Then \(\Exp\qty[\theta]=\dfrac{\alpha}{\alpha+\beta}\).
Proof. \[ \begin{split} \Exp\qty[\theta]&=\int_0^1\theta\cdot\pdfbeta(\alpha,\beta)\dl3\theta=\int_0^1\theta\cdot c\cdot\theta^{\alpha-1}(1-\theta)^{\beta-1}\dl3\theta\\ &=\int_0^1c\cdot\theta^{\alpha}(1-\theta)^{\beta-1}\dl3\theta=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}\frac{\Gamma(\alpha+1)\Gamma(\beta)}{\Gamma(\alpha++\beta+1)}\\ &=\frac{\Gamma(\alpha+1)\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\alpha+\beta+1)}=\frac{\alpha}{\alpha+\beta}. \end{split} \]
6.2 Binomial and Beta
If the prior distribution is Beta, and the sampling data is binomial, then the posterior is also Beta.
Theorem 6.2 Let the prior distribution for \(\theta\) be \[ \Pr(\theta)=\pdfbeta(\alpha, \beta)=c\,\theta^{\alpha-1}(1-\theta)^{\beta-1} \] and the sampling data for \(y\) be \[ \Pr\qty(y\mid\theta)=\dbinom{n}{y}\theta^y(1-\theta)^{n-y}, \] then the posterior distribution is again beta: \[ \Pr\qty(\theta\mid y)=\pdfbeta(\alpha+y, \beta+n-y). \]
Proof. \[ \begin{split} \Pr\qty(\theta\mid y)&=\frac{\Pr\qty(y\mid\theta)\Pr\qty(\theta)}{\int_0^1\Pr\qty(y\mid\theta)\Pr\qty(\theta)\dl3\theta}=c\,\theta^y(1-\theta)^{n-y}\theta^{\alpha-1}(1-\theta)^{\beta-1}\\ &=c\,\theta^{y+\alpha-1}(1-\theta)^{n-y+\beta-1}\\ &=\pdfbeta(\alpha+y,\beta+n-y). \end{split} \]