Chapter 3 Random Vectors and Multivariate Normal …

1 Chapter 3 Random Vectors and MultivariateNormal Random vectorsDefinition Random Vectors are Vectors of random83 BIOS 2083 Linear ModelsAbdus S. Wahedvariables. For instance,X= ,where each element represent a Random variable, is a Random Mean and covariance matrix of a Random mean (expectation) and covariance matrix of a Random vectorXis de-fined as follows:E[X]= E[X1]E[X2]..E[Xn] ,andcov(X)=E {X E(X)}{X E(X)}T = 21 1n 21 n1 2n ,( )where 2j=var(Xj)and jk=cov(Xj,Xk)forj, k=1,2.

2 , 384 BIOS 2083 Linear ModelsAbdus S. WahedProperties of Mean and IfXandYare Random Vectors andA,B,CandDare constant matrices,thenE[AXB+CY+D]=AE[X]B+CE[Y]+D. ( ) as an For any Random vectorX, the covariance matrixcov(X) is as an IfXj,j=1,2,..,nare independent Random variables, thencov(X)=diag( 2j,j=1,2,..,n). as an (X+a)=cov(X) for a constant as an 385 BIOS 2083 Linear ModelsAbdus S. WahedProperties of Mean and Covariance (cont.) (AX)=Acov(X)ATfor a constant as an (X) is positive as an (X)=E[XXT] E[X]{E[X]} as an 386 BIOS 2083 Linear ModelsAbdus S.

3 WahedDefinition correlation matrix of a vector of Random variableXis defined as thematrix of pairwise correlations between the elements ofX. Explicitly,corr(X)= 1 1n n1 ,( )where jk=corr(Xj,Xk)= jk/( j k),j,k=1,2,.., only successive Random variables in the Random vectorXare correlated and have the same correlation , then the correlation matrixcorr(X)isgivenbycorr(X)= 1 1 ..00 ,( ) Chapter 387 BIOS 2083 Linear ModelsAbdus S. WahedExample every pair of Random variables in the Random vectorXhave the same correlation , then the correlation matrixcorr(X)isgivenbycorr(X)= 1.

4 1 ..1 ,( )and the Random variables are said to be Multivariate Normal DistributionDefinition Multivariate Normal Random vectorX=(X1,X2,..,Xn)Tis said to follow a Multivariate Normal distributionwith mean and covariance matrix ifXcanbeexpressedasX=AZ+ ,where =AATandZ=(Z1,Z2,..,Zn) withZi,i=1,2,..,niidN(0,1) 388 BIOS 2083 Linear ModelsAbdus S. WahedBivariate Normal distribution with mean (0,0)Tand covariance matrix 3 2 10123 DensityDefinition Multivariate Normal Random vectorX=(X1,X2,..,Xn)Tis said to follow a Multivariate Normal distributionwith mean and a positive definite covariance matrix ifXhas the densityfX(x)=1(2 )n/2| |1/2exp 12(x )T 1(x ) ( ).

5 Chapter 389 BIOS 2083 Linear ModelsAbdus S. WahedProperties1. Moment generating function of aN( , ) Random variableXis givenbyMX(t)=exp Tt+12tT t .( ) (X)= andcov(X)= .3. IfX1,X2,..,Xnare (0,1) Random variables, then their jointdistribution can be characterized byX=(X1,X2,..,Xn)T N(0,In). Nn( , ) if and only if all non-zero linear combinations of thecomponents ofXare normally 390 BIOS 2083 Linear ModelsAbdus S. WahedLinear transformation5. IfX Nn( , )andAm nis a constant matrix of rankm,thenY=Ax Np(A ,A AT). definition or property 1 linear transformation6.

6 IfX Nn( ,In)andAn nis an orthogonal matrix and =In,thenY=Ax Nn(A ,In). Chapter 391 BIOS 2083 Linear ModelsAbdus S. WahedMarginal and Conditional distributionsSupposeXisNn( , )andXis partitioned as follows,X= X1X2 ,whereX1is of dimensionp 1andX2is of dimensionn p 1. Supposethe corresponding partitions for and are given by = 1 2 ,and = 11 12 21 22 respectively. Then, distribution. X1is Multivariate Normal -Np( 1, 11). the result from property 5 distribution ofX1|X2is p-variate nor-mal -Np( 1|2, 1|2), where, 1|2= 1+ 12 122(X2 2),and 1|2= 11 12 122 21,provided is positive Result , page 156 (Ravishanker and Dey).

7 Chapter 392 BIOS 2083 Linear ModelsAbdus S. WahedUncorrelated implies independence for Multivariate Normal Random vari-ables9. IfX, ,and are partitioned as above, thenX1andX2are independentif and only if 12=0= will use to prove this result. Two Random vectorsX1andX2are independent iffM(X1,X2)(t1,t2)=MX1(t1)MX2(t2).Chapte r 393 BIOS 2083 Linear ModelsAbdus S. Non-central distributionsWe will start with the standard chi-square Chi-square ,X2,..,Xnbeninde-pendentN(0,1) variables, then the distribution of ni=1X2iis 2n(ch-squarewith degrees of freedomn).

8 2n-distribution is a special case of gamma distribution when the scaleparameter is set to 1/2 and the shape parameter is set to ben/2. That is,the density of 2nis given byf 2n(x)=(1/2)n/2 (n/2)e x/2xn/2 1,x 0;n=1,2,..,.( )Example distribution of (n 1)S2/ 2,whereS2= ni=1(Xi X)2/(n 1) is the sample variance of a Random sample of sizenfrom a normaldistribution with mean and variance 2, follows a 2n moment generating function of a chi-square distribution given byM 2n(t)=(1 2t) n/2,t<1/2.( )The ( ) shows that the sum of two independent ch-square randomvariables is also a ch-square.

9 Therefore, differences of sequantial sums ofsquares of independent Normal Random variables will be distributed indepen-dently as 394 BIOS 2083 Linear ModelsAbdus S. WahedTheorem Nn( , )and is positive definite, then(X )T 1(X ) 2n.( ) is positive definite, there exists a non-singularAn nsuch that =AAT( cholesky decomposition ). Then, by definition of multivariatenormal distribution,X=AZ+ ,whereZis a Random sample from aN(0,1) distribution. Now, Chapter 395 BIOS 2083 Linear ModelsAbdus S. =0 =2 =4 =6 =8 =10 Figure : Non-central chi-square densities with df 5 and non-centrality parameter.

10 Definition Non-central chi-square sare as in Definition ( ) except that eachXihas mean i,i=1,2,.., , suppose,X=(X1,..,Xn)Tbe a Random vector distributedasNn( ,In), where =( 1,.., n)T. Then the distribution of ni=1X2i=XTXis referred to as non-central chi-square with non-centralityparameter = ni=1 2i/2=12 T . The density of such a non-central chi-square variable 2n( ) can be written as a infinite poisson mixture of centralchi-square densities as follows:f 2n( )(x)= j=1e jj!(1/2)(n+2j)/2 ((n+2j)/2)e x/2x(n+2j)/2 1.( ) Chapter 396 BIOS 2083 Linear ModelsAbdus S.