Chapter 3 Random Vectors and Multivariate Normal …

1 Chapter 3 Random Vectors and MultivariateNormal Random vectorsDefinition Random Vectors are Vectors of random83 BIOS 2083 Linear ModelsAbdus S. Wahedvariables. For instance,X= ,where each element represent a Random variable, is a Random Mean and covariance matrix of a Random mean (expectation) and covariance matrix of a Random vectorXis de-fined as follows:E[X]= E[X1]E[X2]..E[Xn] ,andcov(X)=E {X E(X)}{X E(X)}T = 21 1n 21 n1 2n ,( )where 2j=var(Xj)and jk=cov(Xj,Xk)forj, k=1,2,.., 384 BIOS 2083 Linear ModelsAbdus S. WahedProperties of Mean and IfXandYare Random Vectors andA,B,CandDare constant matrices,thenE[AXB+CY+D]=AE[X]B+CE[Y]+D. ( ) as an For any Random vectorX, the covariance matrixcov(X) is as an IfXj,j=1,2.

2 ,nare independent Random variables, thencov(X)=diag( 2j,j=1,2,..,n). as an (X+a)=cov(X) for a constant as an 385 BIOS 2083 Linear ModelsAbdus S. WahedProperties of Mean and Covariance (cont.) (AX)=Acov(X)ATfor a constant as an (X) is positive as an (X)=E[XXT] E[X]{E[X]} as an 386 BIOS 2083 Linear ModelsAbdus S. WahedDefinition correlation matrix of a vector of Random variableXis defined as thematrix of pairwise correlations between the elements ofX. Explicitly,corr(X)= 1 1n n1 ,( )where jk=corr(Xj,Xk)= jk/( j k),j,k=1,2,.., only successive Random variables in the Random vectorXare correlated and have the same correlation , then the correlation matrixcorr(X)isgivenbycorr(X)= 1 1 ..00 ,( ) Chapter 387 BIOS 2083 Linear ModelsAbdus S.

3 WahedExample every pair of Random variables in the Random vectorXhave the same correlation , then the correlation matrixcorr(X)isgivenbycorr(X)= 1 .. 1 ..1 ,( )and the Random variables are said to be Multivariate Normal DistributionDefinition Multivariate Normal Random vectorX=(X1,X2,..,Xn)Tis said to follow a Multivariate Normal distributionwith mean and covariance matrix ifXcanbeexpressedasX=AZ+ ,where =AATandZ=(Z1,Z2,..,Zn) withZi,i=1,2,..,niidN(0,1) 388 BIOS 2083 Linear ModelsAbdus S. WahedBivariate Normal distribution with mean (0,0)Tand covariance matrix 3 2 10123 DensityDefinition Multivariate Normal Random vectorX=(X1,X2,..,Xn)Tis said to follow a Multivariate Normal distributionwith mean and a positive definite covariance matrix ifXhas the densityfX(x)=1(2 )n/2| |1/2exp 12(x )T 1(x ) ( ).

4 Chapter 389 BIOS 2083 Linear ModelsAbdus S. WahedProperties1. Moment generating function of aN( , ) Random variableXis givenbyMX(t)=exp Tt+12tT t .( ) (X)= andcov(X)= .3. IfX1,X2,..,Xnare (0,1) Random variables, then their jointdistribution can be characterized byX=(X1,X2,..,Xn)T N(0,In). Nn( , ) if and only if all non-zero linear combinations of thecomponents ofXare normally 390 BIOS 2083 Linear ModelsAbdus S. WahedLinear transformation5. IfX Nn( , )andAm nis a constant matrix of rankm,thenY=Ax Np(A ,A AT). definition or property 1 linear transformation6. IfX Nn( ,In)andAn nis an orthogonal matrix and =In,thenY=Ax Nn(A ,In). Chapter 391 BIOS 2083 Linear ModelsAbdus S. WahedMarginal and Conditional distributionsSupposeXisNn( , )andXis partitioned as follows,X= X1X2 ,whereX1is of dimensionp 1andX2is of dimensionn p 1.

5 Supposethe corresponding partitions for and are given by = 1 2 ,and = 11 12 21 22 respectively. Then, distribution . X1is Multivariate Normal -Np( 1, 11). the result from property 5 distribution ofX1|X2is p-variate nor-mal -Np( 1|2, 1|2), where, 1|2= 1+ 12 122(X2 2),and 1|2= 11 12 122 21,provided is positive Result , page 156 (Ravishanker and Dey). Chapter 392 BIOS 2083 Linear ModelsAbdus S. WahedUncorrelated implies independence for Multivariate Normal Random vari-ables9. IfX, ,and are partitioned as above, thenX1andX2are independentif and only if 12=0= will use to prove this result. Two Random vectorsX1andX2are independent iffM(X1,X2)(t1,t2)=MX1(t1)MX2(t2).Chapte r 393 BIOS 2083 Linear ModelsAbdus S. Non-central distributionsWe will start with the standard chi-square Chi-square ,X2.

6 ,Xnbeninde-pendentN(0,1) variables, then the distribution of ni=1X2iis 2n(ch-squarewith degrees of freedomn). 2n- distribution is a special case of gamma distribution when the scaleparameter is set to 1/2 and the shape parameter is set to ben/2. That is,the density of 2nis given byf 2n(x)=(1/2)n/2 (n/2)e x/2xn/2 1,x 0;n=1,2,..,.( )Example distribution of (n 1)S2/ 2,whereS2= ni=1(Xi X)2/(n 1) is the sample variance of a Random sample of sizenfrom a normaldistribution with mean and variance 2, follows a 2n moment generating function of a chi-square distribution given byM 2n(t)=(1 2t) n/2,t<1/2.( )The ( ) shows that the sum of two independent ch-square randomvariables is also a ch-square. Therefore, differences of sequantial sums ofsquares of independent Normal Random variables will be distributed indepen-dently as 394 BIOS 2083 Linear ModelsAbdus S.

7 WahedTheorem Nn( , )and is positive definite, then(X )T 1(X ) 2n.( ) is positive definite, there exists a non-singularAn nsuch that =AAT(Cholesky decomposition). Then, by definition of multivariatenormal distribution ,X=AZ+ ,whereZis a Random sample from aN(0,1) distribution . Now, Chapter 395 BIOS 2083 Linear ModelsAbdus S. =0 =2 =4 =6 =8 =10 Figure : Non-central chi-square densities with df 5 and non-centrality parameter .Definition Non-central chi-square sare as in Definition ( ) except that eachXihas mean i,i=1,2,.., , suppose,X=(X1,..,Xn)Tbe a Random vector distributedasNn( ,In), where =( 1,.., n)T. Then the distribution of ni=1X2i=XTXis referred to as non-central chi-square with non-centralityparameter = ni=1 2i/2=12 T . The density of such a non-central chi-square variable 2n( ) can be written as a infinite poisson mixture of centralchi-square densities as follows:f 2n( )(x)= j=1e jj!

8 (1/2)(n+2j)/2 ((n+2j)/2)e x/2x(n+2j)/2 1.( ) Chapter 396 BIOS 2083 Linear ModelsAbdus S. WahedProperties1. The moment generating function of anon-central chi-square variable 2n( )isgivenbyM 2n(n, )(t)=(1 2t) n/2exp 2 t1 2t ,t<1/2.( ) 2n( ) =n+2 . 2n( ) =2(n+4 ).4. 2n(0) For a given constantc,(a)P( 2n( )>c) is an increasing function of .(b)P( 2n( )>c) P( 2n>c). Chapter 397 BIOS 2083 Linear ModelsAbdus S. WahedTheorem Nn( , )and is positive definite, thenXT 1X 2n( = T 1 /2).( ) is positive definite, there exists a non-singular matrixAn nsuch that =AAT(Cholesky decomposition). Define,Y={AT} , Chapter 398 BIOS 2083 Linear ModelsAbdus S. =0 =2 =4 =6 =8 =10 Figure : Non-central F-densities with df 5 and 15 and non-centrality parameter .Definition 2n1( )andU2 2n2andU1andU2are independent, then, the distribution ofF=U1/n1U2/n2( )is referred to as non-centralF- distribution with dfn1andn2, and non-centrality parameter.

9 Chapter 399 BIOS 2083 Linear ModelsAbdus S. Wahed 505101520 =0 =2 =4 =6 =8 =10 Figure : Non-central t-densities with df 5 and non-centrality parameter .Definition N( ,1) andU2 2nandU1andU2are independent, then, the distribution ofT=U1 U2/n( )is referred to as non-centralt- distribution with dfnand non-centrality pa-rameter . Chapter 3100 BIOS 2083 Linear ModelsAbdus S. distribution of quadratic formsCaution: We assume that our matrix of quadratic form is nis symmetric and idempotent with rankr,thenrofits eigenvalues are exactly equal to1andn rare equal to spectral decomposition theorem. (See Result on page 51 ofRavishanker and Dey).Theorem Nn(0,In). The quadratic formXTAX 2riffAis idempotent withrank(A)= (symmetric) idempotent matrix of rankr. Then, by spectraldecomposition theorem, there exists an orthogonal matrixPsuch thatPTAP= = Ir000.

10 ( )DefineY=PTX= PT1 XPT2X = Y1Y2 ,sothatPT1P1= ,X= Chapter 3101 BIOS 2083 Linear ModelsAbdus S. WahedPYandY1 Nr(0,Ir). Now,XTAx=(PY)TAPY=YT Ir000 Y=YT1Y1 2r.( )Now supposeXTAX 2r. This means that the moment generatingfunction ofXTAXis given byMXTAX(t)=(1 2t) r/2.( )But, one can calculate the ofXTAX directly using the multivariatenormal density asMXTAX(t)=E exp (XTAX)t = exp (XTAX)t fX(x)dx= exp (XTAX)t 1(2 )n/2exp 12xTx dx= 1(2 )n/2exp 12xT(In 2tA)x dx=|In 2tA| 1/2=n i=1(1 2t i) 1/2.( )Equate ( ) and ( ) to obtain the desired 3102 BIOS 2083 Linear ModelsAbdus S. WahedTheorem Nn( , )where is positive definite. The quadraticformXTAX 2r( )where = TA /2,iffA is idempotent withrank(A )= of two quadratic Nn( , )where is positive definite. The two quadratic formsXTAXandXTBXare independent if and only ifA B=0=B A.