第7周概率课小结

1. Covariance(协方差)

$X$与$Y$的Covariance: $$\sigma_{XY} = Cov(X, Y) = E_{XY}[(X - \mu_X)(Y - \mu_Y)]$$

将式子展开可得: $$Cov(X, Y) = E_{XY}(XY) - E_X(X)E_Y(Y)$$

2. Uncertainty Principle

$$\sigma_{XY}^2 \leq \sigma_X^2 \sigma_Y^2$$

证明:

已知:$E_{XY}[(X - \mu_X) - c(Y - \mu_Y))^2] \geq 0, \forall c \in R$
展开可得: $E_{XY}[(X - \mu_X)^2 + c^2 (Y - \mu_Y)^2 - 2c(X - \mu_X)(Y - \mu_Y)] \geq 0$
由数学期望线性性质:$E_{X}[(X - \mu_X)^2] + c^2 E_{Y}[(Y - \mu_Y)^2] - 2cE_{XY}[(X - \mu_X)(Y - \mu_Y)] \geq 0$
得:$\sigma_X^2 + c^2 \sigma_Y^2 - 2c \sigma_{XY} \geq 0$
因为$c$可取任意实数,令$c = \frac{\sigma_{XY}}{\sigma_Y^2}$,带入上式,可得:
$\sigma_X^2 + \frac{\sigma_{XY}^2}{\sigma_Y^2} - 2\frac{\sigma_{XY}^2}{\sigma_Y^2} \geq 0$
即$\sigma_X^2 - \frac{\sigma_{XY}^2}{\sigma_Y^2} \geq 0$
整理即得: $\sigma_{XY}^2 \leq \sigma_X^2 \sigma_Y^2$,得证

3. Correlation (相关系数)

  • $X$与$Y$的Correlation: $$\rho_{XY} = \frac{Cov(X, Y)}{\sqrt{Var(X)}\sqrt{Var(Y)}} = \frac{\sigma_{XY}}{\sigma_X \sigma_Y}$$

  • 若两个随机变量独立,则correlation = 0; 反之不一定成立(对高斯分布的随机变量是成立的)

  • 如果对上面的uncertainty principle稍作变换,即可得到correlation的一个取值范围:

\begin{align} &\Rightarrow |\sigma_{XY}| \leq \sigma_X \sigma_Y \\ & \Rightarrow -\sigma_X\sigma_Y \leq \sigma_{XY} \leq \sigma_X\sigma_Y \\\ & \Rightarrow -1 \leq \rho_{XY} \leq 1 \end{align}

4. Bivariate (二维随机变量)

  • 设$(X, Y)$ 是二维随机变量,对于任意实数$x, y$,二元函数 $$ F(x, y) = P \{ (X \leq x) \cap (Y \leq y) \} \stackrel{记作}{=} P(X \leq x, Y \leq y)$$称为二维随机变量$(X, Y)$的分布函数(cdf), 或称为随机变量$X$和$Y$的联合分布函数(joint cdf)

  • 对于$F(x, y)$,若存在非负可积函数$f(x,y)$使对于任意$x, y$有$$F(x,y) = \int_{-\infty}^y \int_{-\infty}^x f(u, v)dudv$$ 则称$(X, Y)$是连续型二维随机变量,函数$f(x, y)$为二维随机变量$(X, Y)$的概率密度(pdf), 或随机变量$X$和$Y$的联合概率密度(joint pdf)

  • (joint) cdf 和 (joint) pdf有以下关系:

    • 条件概率密度 (conditional pdf) $$f(y | x) = \frac{f(x, y)}{f(x)}$$

    • 边缘概率密度 (marginal pdf) $$f(x) = \int_{-\infty}^{+\infty}f(x, y)dy$$

    • cdf $\rightarrow$ pdf $$f(x, y) = \frac{\partial^2}{\partial x \partial y}F(x,y)$$

    • 边缘分布函数 (marginal cdf) $$F(x) = \lim_{y \rightarrow \infty}F(x, y)$$

    • 条件分布函数 (conditional cdf) $$F(y|x) = \frac{F(x, y)}{F(x)}$$

5. Cross-Covariance Matrix & Covariance Matrix(协方差矩阵)

  • 对$m$维随机变量 $X = [X_1, X_2, …, X_m]^T$ 和 $n$维随机变量 $Y = [Y_1, Y_2, …, Y_n]^T$ $$K_{XY} = E[(X - \mu_X)(Y - \mu_Y)^T]$$ 为$X$和$Y$的的cross-covariance matrix.

  • 若$Y = X$, 则可得$X$的covariance matrix: $$K_{X} = E[(X - \mu_X)(X - \mu_X)^T]$$ 这是一个对称矩阵

6. Multivariate Gaussian distribution

  • 若$n$维随机变量 $X = [X_1, X_2, …, X_d]^T$为multivariate gaussian/normal distribution或称为jointly gaussian, 且期望为$\mu_X = [\mu_{X_1}, \mu_{X_2}\, …, \mu_{X_d}]^T$, covariance matrix为 $K_X = E[(X - \mu_{\small X})(X - \mu_{\small X})^T]$, 则其pdf为 $$f(X) = \frac{1}{(2\pi)^\frac{d}{2}(det(K_X))^\frac{1}{2}}\Large{e^{-\frac{(X - \mu_{\small X})^{\small T} K^{\small -1} (X - \mu_{\small X})}{2}}}$$

参考文献

  1. 盛骤. 谢式千. 潘承毅《概率论与数理统计》第四版,高等教育出版社
  2. 503课堂笔记
  3. 503讨论课笔记
comments powered by Disqus