Multivariate Gaussian Distribution - Mathematics Home

1 Multivariate Gaussian DistributionThe random vectorX= (X1,X2,..,Xp)is said to have amultivariate Gaussian distributionif the joint distributionofX1,X2,..,Xphas densityfX(x1,x2,..,xp) =1(2 )p/2det( )1/2exp( 12(x )t 1(x ))(1)where is ap psymmetric, positive definite matrix. The notation is asfollows:xis the column vectorx= , is the column vector = 1 p , 1is the inverse of the matrix andtdenotes matrix transposition. Thusthe quantity appearing in the exponential is a 1 pmatrix times ap pmatrix times ap 1 matrix; and hence, a 1 1 matrix, a real (x )t 1(x ) =p k,`=1(xk k) 1k`(x` `)where 1k`is the (k,`)th matrix element of 1. The constants in front ofthe exponential are normalization constants; that is, if (1) is integrated overRpthen the result equals 1.

2 The vector is the mean vector sinceE(X) = 1as an exercise in integration shows. It is convenient for theoretical purposesto centerX; that is, ifE(X)6= 0, the replaceXbyX . From now on weassumeE(X) = 0 in which case the Multivariate Gaussian (1) becomesfX(x1,x2,..,xp) =1(2 )p/2det( )1/2exp( 12xt 1x)(2)Now the matrixXXtis ap pmatrix with elementsXiXj. (NoteXtXis1 1 butXXtisp p.). One can show (by evaluating integrals) that (recallwe are setting = 0)E(XXt) = ,that is,E(XiXj) = ij. The matrix is called thecovariance Remark:If the covariance matrix is diagonal, then the den-sityfXfactors and the random variables are p=2 caseWe examine the casep= 2 in more detail. That is we have a random vectorX= (X1,X2) whose Distribution is given by (2) forp= 2.

3 In this case it iscustomary to parametrize (for reasons that will become clear) as follows: =( 21 1 2 1 2 22).Sincedet = 21 22(1 2)and det >0 (recall is positive definite), we must have 1< < coefficient is called thecorrelation coefficientsince when it equals 0 therandom variablesX1andX2are independent. A calculation of the inverseof gives 1=(1 21(1 2) 1 2(1 2) 1 2(1 2)1 22(1 2)).2 Substituting this into (2) gives thebivariate normal densityfX(x1,x2) =12 1 2 1 2exp( 12(1 2)(x1 21 2 x1x2 1 2+x22 22))(3)Recall the integral e ax2+2bxdx= aeb2/a,a>0.(4)Using this it is easy to show that the marginal densities are gaussiansfX1(x1) = fX(x1,x2)dx2=1 2 1e x21/(2 21),fX2(x2) = fX(x1,x2)dx1=1 2 2e x22/(2 22).

4 (5)We now calculateE(X1|X2), the expected value ofX1givenX2. To do thiswe first find the conditional densityf(x1|x2). It is, by definition,f(x1|x2) =fX(x1,x2)fX2(x2)wherefXis given by (3) andfX2by (5). Carrying out the algebra we seethe conditional density is of the formf(x1|x2) =de ax21+2bx1x2 cx22wherea=12(1 2) 21b= 2(1 2) 1 2c= 22(1 2) 22d=1 2 (1 2) 13 ThenE(X1|X2) = x1f(x1|x2) last integral can be computed using (4) with the result thatE(X1|X2) = ( 1 2) X2.(6)Remarks:1. If = 0 thenX1andX2are independent andE(X1) =E(X1|X2) = If >0 then the expected value ofX1givenX2is positively correlatedwithX2. (Similarly, it is negatively correlated when <0.)3. The conditional expectation