Beta分布和Dirichlet分布

　　在《Gamma函数是如何被发现的？》里证明了\begin{align*} B(m, n) = \int_0^1 x^{m-1} (1-x)^{n-1} \text{d} x = \frac{\Gamma (m) \Gamma (n)}{\Gamma (m+n)} \end{align*}于是令\begin{align*} f_{m,n}(x) = \begin{cases} \frac{x^{m-1} (1-x)^{n-1}}{B(m, n)} = \frac{\Gamma (m+n)}{\Gamma (m) \Gamma (n)} x^{m-1} (1-x)^{n-1} & 0 \leq x \leq 1 \\ 0 & \text{其他情况} \end{cases} \end{align*}可知$f_{m,n}(x)$积分为$1$，即$f_{m,n}(x)$对应着某个概率分布，由于这个函数的分母是Beta函数，我们一般称其对应的分布是参数为$m,n$的Beta分布。

　　下面简单给出该分布的数字特征，易知其$k$阶矩为\begin{align*} E[x^k] = \int_0^1 x^k f_{m,n}(x) \text{d} x = \int_0^1 \frac{x^{m + k -1} (1-x)^{n-1}}{B(m + k, n)} \frac{B(m + k, n)}{B(m, n)}\text{d} x = \frac{\Gamma (m + k) \Gamma (m+n)}{\Gamma (m)\Gamma (m+ k +n)} \end{align*}于是\begin{align*} E[x] = \frac{\Gamma (m + 1) \Gamma (m+n)}{\Gamma (m)\Gamma (m+ 1 +n)} = \frac{m}{m+n}, \ E[x^2] = \frac{\Gamma (m + 2) \Gamma (m+n)}{\Gamma (m)\Gamma (m+ 2 +n)} = \frac{(m+1)m}{(m+n+1)(m+n)} \end{align*}故其均值和方差分别为\begin{align*} E[x] = \frac{m}{m+n}, \ D[x] = \frac{(m+1)m}{(m+n+1)(m+n)} - \left(\frac{m}{m+n}\right)^2 = \frac{mn}{(m+n+1)(m+n)^2} \end{align*}

　　Beta函数是二元的，可将其推广成如下$k+1(k \geq 2)$元的形式：\begin{align} \label{eq: multivariate beta function} B(m_1, \cdots, m_{k+1}) = \int_0^1 x_1^{m_1-1} \int_0^{1-x_1} x_2^{m_2-1} \cdots  \int_0^{1-x_1 - \cdots - x_{k-1}} x_k^{m_k-1} (1 - x_1 - \cdots - x_k)^{m_{k+1}-1} \text{d} x_1 \text{d} x_2 \cdots \text{d} x_k \end{align}注意式(\ref{eq: multivariate beta function})是一个$k$重积分，考察最里面对$x_k$的积分，即\begin{align*} E_k(m_k, m_{k+1}) = \int_0^{1-x_1 - \cdots - x_{k-1}} x_k^{m_k-1} (1 - x_1 - \cdots - x_k)^{m_{k+1}-1} \text{d} x_k = \int_0^t x_k^{m_k-1} (t - x_k)^{m_{k+1}-1} \text{d} x_k \end{align*}其中$t = 1-x_1 - \cdots - x_{k-1}$。由分部积分易知有\begin{align*} E_k(m_k, m_{k+1}) & = \int_0^t (t - x_k)^{m_{k+1}-1} \text{d} \frac{x_k^{m_k}}{m_k} \\ & = (t - x_k)^{m_{k+1}-1} \frac{x_k^{m_k}}{m_k} |_0^t - \int_0^t \frac{x_k^{m_k}}{m_k} (m_{k+1}-1) (t - x_k)^{m_{k+1}-2} (-1) \text{d} x_k \\ & = \frac{m_{k+1}-1}{m_k} E_k(m_k+1, m_{k+1}-1) \end{align*}于是递推下去有\begin{align*} E_k(m_k, m_{k+1}) & = \frac{m_{k+1}-1}{m_k} E_k(m_k+1, m_{k+1}-1) \\ & = \frac{m_{k+1}-1}{m_k} \frac{m_{k+1}-2}{m_k+1} E_k(m_k+2, m_{k+1}-2) \\ & = \cdots \\ & = \frac{m_{k+1}-1}{m_k} \cdots \frac{1}{m_k + m_{k+1} - 2} E_k(m_k + m_{k+1} - 1, 1) \end{align*}又\begin{align*} E_k(m_k + m_{k+1} - 1, 1) = \int_0^t x_k^{m_k + m_{k+1} - 2} \text{d} x_k = \frac{x_k^{m_k + m_{k+1} - 1}}{m_k + m_{k+1} - 1} |_0^t = \frac{t^{m_k + m_{k+1} - 1}}{m_k + m_{k+1} - 1} \end{align*}于是\begin{align*} E_k(m_k, m_{k+1}) = \frac{\Gamma(m_{k+1}) \Gamma(m_k)}{\Gamma(m_{k+1} + m_k)} (1-x_1 - \cdots - x_{k-1})^{m_k + m_{k+1} - 1} \end{align*}将其回代入式(\ref{eq: multivariate beta function})，接着考察最里面对$x_{k-1}$的积分\begin{align*} E_{k-1}(m_{k-1}, m_k + m_{k+1}) & = \int_0^{1-x_1 - \cdots - x_{k-2}} x_{k-1}^{m_{k-1}-1} \frac{\Gamma(m_{k+1}) \Gamma(m_k)}{\Gamma(m_{k+1} + m_k)} (1-x_1 - \cdots - x_{k-1})^{m_k + m_{k+1} - 1} \text{d} x_{k-1} \\ & = \frac{\Gamma(m_{k+1}) \Gamma(m_k)}{\Gamma(m_{k+1} + m_k)} \int_0^t x_{k-1}^{m_{k-1}-1}  (t - x_{k-1})^{m_k + m_{k+1} - 1} \text{d} x_{k-1} \end{align*}其中$t = 1-x_1 - \cdots - x_{k-2}$。于是继续仿照前面的方法(分部积分后递推)可得\begin{align*} E_{k-1}(m_{k-1}, m_k + m_{k+1}) & = \frac{\Gamma(m_{k+1}) \Gamma(m_k)}{\Gamma(m_{k+1} + m_k)} \frac{\Gamma(m_{k+1} + m_k) \Gamma(m_{k-1})}{\Gamma(m_{k+1} + m_k + m_{k-1})} (1-x_1 - \cdots - x_{k-2})^{m_{k+1} + m_k + m_{k-1} - 1} \\ & = \frac{\Gamma(m_{k+1}) \Gamma(m_k) \Gamma(m_{k-1})}{\Gamma(m_{k+1} + m_k + m_{k-1})} (1-x_1 - \cdots - x_{k-2})^{m_{k+1} + m_k + m_{k-1} - 1} \end{align*}不断重复这个过程可知\begin{align} \label{eq: E2} E_2(m_2, m_{k+1} + m_k + \cdots + m_3) = \frac{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_2)}{\Gamma(m_{k+1} + m_k + \cdots + m_2)} (1-x_1)^{m_{k+1} + m_k + \cdots + m_2 - 1} \end{align}于是最终对$x_1$的积分为\begin{align*} B(m_1, \cdots, m_{k+1}) & = \int_0^1 x_1^{m_1-1} \frac{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_2)}{\Gamma(m_{k+1} + m_k + \cdots + m_2)} (1-x_1)^{m_{k+1} + m_k + \cdots + m_2 - 1} \text{d} x_1 \\ & = \frac{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_2)}{\Gamma(m_{k+1} + m_k + \cdots + m_2)} \frac{\Gamma(m_{k+1} + m_k + \cdots + m_2) \Gamma(m_1)}{\Gamma(m_{k+1} + m_k + \cdots + m_1)} 1^{m_{k+1} + m_k + \cdots + m_1 - 1} \\ & = \frac{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_1)}{\Gamma(m_{k+1} + m_k + \cdots + m_1)} \end{align*}令$\boldsymbol{m} = [m_1, \cdots, m_{k+1}]$，$\boldsymbol{x} = [x_1, \cdots, x_{k+1}]$且定义\begin{align*} f_{\boldsymbol{m}} (\boldsymbol{x}) = \begin{cases} \frac{\Gamma(m_{k+1} + m_k + \cdots + m_1)}{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_1)} \prod_{i=1}^{k+1} x_i^{m_i - 1} & \sum_{i=1}^{k+1} x_i = 1 \\ 0 & \text{其他情况}\end{cases} \end{align*}注意这是一个$k$变量的函数(和为$1$的限制)，由上面的推导可知$f_{\boldsymbol{m}} (\boldsymbol{x})$的$k$重积分为$1$，故$f_{\boldsymbol{m}} (\boldsymbol{x})$也对应着某个概率分布，我们称其对应的分布是参数为$\boldsymbol{m}$的Dirichlet分布。

　　下面简单给出该分布的数字特征，易知\begin{align*} x_j^n f_{\boldsymbol{m}} (\boldsymbol{x}) & = \frac{\Gamma(m_{k+1} + \cdots + m_1)}{\Gamma(m_{k+1}) \cdots \Gamma(m_1)} x_j^n \prod_{i=1}^{k+1} x_i^{m_i - 1} \\ & = \frac{\Gamma(m_{k+1} + \cdots + m_1)}{\Gamma(m_{k+1} + \cdots + m_j + n + \cdots + m_1)} \frac{\Gamma(m_j + n)}{\Gamma(m_j)} \frac{\Gamma(m_{k+1} + \cdots + m_j + n + \cdots + m_1)}{\Gamma(m_{k+1}) \cdots \Gamma(m_j + n) \cdots \Gamma(m_1)}  x_j^n \prod_{i=1}^{k+1} x_i^{m_i - 1} \end{align*}于是
\begin{align*} E[x_j] & = \frac{\Gamma(m_{k+1} + \cdots + m_1)}{\Gamma(m_{k+1} + \cdots + m_j + 1 + \cdots + m_1)} \frac{\Gamma(m_j + 1)}{\Gamma(m_j)} = \frac{m_j}{m_{k+1} + \cdots + m_1} \\ E[x_j^2] & = \frac{\Gamma(m_{k+1} + \cdots + m_1)}{\Gamma(m_{k+1} + \cdots + m_j + 2 + \cdots + m_1)} \frac{\Gamma(m_j + 2)}{\Gamma(m_j)} = \frac{(m_j+1)m_j}{(m_{k+1} + \cdots + m_1 + 1)(m_{k+1} + \cdots + m_1)} \end{align*}故其均值和方差分别为\begin{align*} E[x_j] & = \frac{m_j}{m_{k+1} + \cdots + m_1} \\ D[x_j] & = \frac{(m_j+1)m_j}{(m_{k+1} + \cdots + m_1 + 1)(m_{k+1} + \cdots + m_1)} - \left(\frac{m_j}{m_{k+1} + \cdots + m_1}\right)^2 = \frac{m_j (m_{k+1} + \cdots + m_1 - m_j)}{(m_{k+1} + \cdots + m_1+1)(m_{k+1} + \cdots + m_1)^2} \end{align*}又\begin{align*} x_p x_q f_{\boldsymbol{m}} (\boldsymbol{x}) & = \frac{\Gamma(m_{k+1} + \cdots + m_1)}{\Gamma(m_{k+1}) \cdots \Gamma(m_1)} x_p x_q \prod_{i=1}^{k+1} x_i^{m_i - 1} \\ & = \frac{\Gamma(m_{k+1} + \cdots + m_1)}{\Gamma(m_{k+1} + \cdots + m_1 + 2)} \frac{\Gamma(m_p + 1)}{\Gamma(m_p)} \frac{\Gamma(m_q + 1)}{\Gamma(m_q)} \frac{\Gamma(m_{k+1} + \cdots + m_1 + 2)}{\Gamma(m_{k+1}) \cdots \Gamma(m_p + 1) \cdots \Gamma(m_q + 1) \cdots \Gamma(m_1)} x_p x_q \prod_{i=1}^{k+1} x_i^{m_i - 1} \end{align*}于是\begin{align*} E[x_p x_q] = \frac{\Gamma(m_{k+1} + \cdots + m_1)}{\Gamma(m_{k+1} + \cdots + m_1 + 2)} \frac{\Gamma(m_p + 1)}{\Gamma(m_p)} \frac{\Gamma(m_q + 1)}{\Gamma(m_q)} = \frac{m_p m_q}{(m_{k+1} + \cdots + m_1 + 1)(m_{k+1} + \cdots + m_1)} \end{align*}故协方差为\begin{align*} cov(x_p, x_q) & = E[x_p x_q] - E[x_p] E[x_q] \\ & = \frac{m_p m_q}{(m_{k+1} + \cdots + m_1 + 1)(m_{k+1} + \cdots + m_1)} - \frac{m_p}{m_{k+1} + \cdots + m_1} \frac{m_q}{m_{k+1} + \cdots + m_1} \\ & = \frac{-m_p m_q}{(m_{k+1} + \cdots + m_1+1)(m_{k+1} + \cdots + m_1)^2} \end{align*}

　　由式(\ref{eq: E2})知\begin{align*} P(x_1 = t) & = t^{m_1 - 1} \frac{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_2)}{\Gamma(m_{k+1} + m_k + \cdots + m_2)} (1-t)^{m_{k+1} + m_k + \cdots + m_2 - 1} \\ & = \frac{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_1)}{\Gamma(m_1) \Gamma(m_{k+1} + m_k + \cdots + m_1 - m_1)} t^{m_1 - 1} (1-t)^{m_{k+1} + m_k + \cdots + m_1 - m_1 - 1} \end{align*}由对称性可知
\begin{align*} P(x_i = t) = \frac{\Gamma(m_{k+1}) \Gamma(m_k) \cdots \Gamma(m_1)}{\Gamma(m_i) \Gamma(m_{k+1} + m_k + \cdots + m_1 - m_i)} t^{m_i - 1} (1-t)^{m_{k+1} + m_k + \cdots + m_1 - m_i - 1} \end{align*}这意味着Dirichlet分布的变量$x_i$的边际分布是参数为$m_i, m_{k+1} + m_k + \cdots + m_1 - m_i$的Beta分布。