We derive the classical result: what is the density of a multivariate normal conditioned on some proper subset of its components?

That is, if

$\mathbf{Pr}(Y_1 = y_1, Y_2 = y_2) = \frac{1}{(2\pi)^{n/2} |\Omega|^{1/2}} \exp \left( -\frac{1}{2} Q(y_1 - \mu_1, y_2 - \mu_2) \right)$

where

$Q(x_1, x_2) = \begin{pmatrix} x_1 & x_2 \end{pmatrix} \begin{bmatrix} A & B \\ B^\intercal & C \end{bmatrix}^{-1} \begin{pmatrix} x_1\\ x_2 \end{pmatrix}$

then we want to characterize

$\mathbf{Pr}(Y_2 = y_2 | Y_1 = y_1) = \frac{ \mathbf{Pr}(Y_1 = y_1, Y_2 = y_2) } { \mathbf{Pr}(Y_1 = y_1) }$

Preliminary Results

We’ll want a couple of preliminary results before establishing the primary result.

Marginal Distribution

Using the moment generating function, it is easy to show that the marginal distribution of $Y_1$ is

$Y_1 \sim N(\mu_1, A )$

Block Matrix Inverse

The key piece of the derivation relies on being able to compute the inverse of a partitioned, 2x2 matrix. To establish a formula for the inverse, use gaussian elimination.

$\begin{bmatrix} I & -A^{-1} B D^{-1} \\ 0 & D^{-1} \end{bmatrix} \begin{bmatrix} A^{-1} & 0 \\ -B^\intercal A^{-1} & I \end{bmatrix} \begin{bmatrix} A & B \\ B^\intercal & C \end{bmatrix} = \begin{bmatrix} I & 0 \\ 0 & I \end{bmatrix}$

where $D = C - B^\intercal A^{-1} B$ .

Hence,

$\begin{bmatrix} A & B \\ B^\intercal & C \end{bmatrix}^{-1} = \begin{bmatrix} A^{-1} + A^{-1} B D^{-1} B^\intercal A^{-1} & -A^{-1} B D^{-1} \\ -D^{-1} B^\intercal A^{-1} & D^{-1} \end{bmatrix}$

Conditional Distribution

Let $\bar{y}_1 = y_1 - \mu_1$ , $\bar{y}_2 = y_2 - \mu_2$ .

Then the quadratic term in $\mathbf{Pr}(Y_1 = y_1, Y_2 = y_2)$ is

$\begin{align*} Q(y_1 - \mu_1, y_2 - \mu_2) & = Q(\bar{y}_1, \bar{y}_2) \\ & = \bar{y}_1^\intercal A^{-1} \bar{y}_1 + \bar{y}_1^\intercal A^{-1} B D^{-1} B^\intercal A^{-1} \bar{y}_1 - 2 \bar{y}_1^\intercal A^{-1} B D^{-1} \bar{y}_2 + \bar{y}_2^\intercal D^{-1} \bar{y}_2 \\ & = \bar{y}_1^\intercal A^{-1} \bar{y}_1 + (\bar{y}_2 - v) ^\intercal D^{-1} (\bar{y}_2 - v) \end{align*}$

where $v^\intercal = \bar{y}_1^\intercal A^{-1} B$ .

In particular, we can write the conditional density as

$\mathbf{Pr}(Y_2 = y_2 | Y_1 = y_1) = c_0 \exp \left( -\frac{1}{2} (\bar{y}_2 - v) ^\intercal D^{-1} (\bar{y}_2 - v) \right)$

Equivalently,

$\left[ Y_2 | Y_1 = y_1 \right] \sim N \left( \mu_2 + B^\intercal A^{-1} (y_1 - \mu_1), C - B^\intercal A^{-1} B \right)$