Multivariate Normal Distribution

Probability Density Function

If X = (X1,…, Xn) is a multivariate normal random vector, denoted X∼N(μ, Σ) or X∼ Nn(μ, Σ), then its density is given by

where μ = (μ1,…, μn) = E(X) is a vector whose and Σ is the nonsingular variance–covariance matrix (n × n) whose diagonal terms are variances and off-diagonal terms are covariances:

[Page 863]

Note that the covariance matrix Σ is symmetric and positive definite. The (i,j)th element is given by

and.

An important special case of the multivariate normal distribution is the bivariate normal. If

where and, then the bivariate density is given by

Let X=(X1,X2)′; the joint density can be rewritten in matrix notation as

Multivariate Normal Density Contours

The contour levels of fX(x), that is, the set of points in

for which fX(x) is constant, satisfy

These surfaces are n-dimensional ellipsoids centered at μ, whose axes of symmetry are given by the principal components (the eigenvectors) of Σ. Specifically, the length of the ellipsoid along the ith axis is

, where λi is the ith eigenvalue associated with the eigenvector ei (recall that eigenvectors ei and eigenvalues λi are solutions to Σei = λiei for i = 1,…, n).

Some Basic Properties

The following list presents some important properties involving the multivariate normal distribution.

1. The first two moments of a multivariate normal distribution, namely μ and Σ, completely characterize the distribution. In other words, if X and Y are both multivariate normal with the same first two moments, then they are similarly distributed.

2. Let X =(X1, …, Xn)′ be a multivariate normal random vector with mean μ and covariance matrix Σ, and let

. The linear combination Y = α′X = α1X1 + … + αnXn is normal with mean E(Y) = α′μ and variance. Also, if α′X is normal with mean α′μ and variance α′Σα for all possible α, then X must be a multivariate normal random vector with mean μ and covariance matrix.

3. More generally, let X =(X1,…, Xn)′ be a multivariate normal random vector with mean μ and covariance matrix Σ, and let

be a full rank matrix with m≤n, the set of linear combinations Y = (Y1,…, Ym)′ = AX is multivariate normally distributed with mean Aμ and covariance matrix AΣA′. Also, if Y = AX + b where b is a m × 1 vector of constants, then Y is multivariate normally distributed with mean Aμ + b and co-variance matrix AΣA′.

...