Chapter 3 Random Vectors and Multivariate Normal …

1 Chapter 3 Random Vectors and MultivariateNormal Random vectorsDefinition Random Vectors are Vectors of random83 BIOS 2083 Linear ModelsAbdus S. Wahedvariables. For instance,X= ,where each element represent a Random variable, is a Random Mean and covariance matrix of a Random mean (expectation) and covariance matrix of a Random vectorXis de-fined as follows:E[X]= E[X1]E[X2]..E[Xn] ,andcov(X)=E {X E(X)}{X E(X)}T = 21 1n 21 n1 2n ,( )where 2j=var(Xj)and jk=cov(Xj,Xk)forj, k=1,2,.., 384 BIOS 2083 Linear ModelsAbdus S. WahedProperties of Mean and IfXandYare Random Vectors andA,B,CandDare constant matrices,thenE[AXB+CY+D]=AE[X]B+CE[Y]+D. ( ) as an For any Random vectorX, the covariance matrixcov(X) is as an IfXj,j=1,2,..,nare independent Random variables, thencov(X)=diag( 2j,j=1,2.)

2 ,n). as an (X+a)=cov(X) for a constant as an 385 BIOS 2083 Linear ModelsAbdus S. WahedProperties of Mean and Covariance (cont.) (AX)=Acov(X)ATfor a constant as an (X) is positive as an (X)=E[XXT] E[X]{E[X]} as an 386 BIOS 2083 Linear ModelsAbdus S. WahedDefinition correlation matrix of a vector of Random variableXis defined as thematrix of pairwise correlations between the elements ofX. Explicitly,corr(X)= 1 1n n1 ,( )where jk=corr(Xj,Xk)= jk/( j k),j,k=1,2,.., only successive Random variables in the Random vectorXare correlated and have the same correlation , then the correlation matrixcorr(X)isgivenbycorr(X)= 1 1 ..00 ,( ) Chapter 387 BIOS 2083 Linear ModelsAbdus S. WahedExample every pair of Random variables in the Random vectorXhave the same correlation , then the correlation matrixcorr(X)isgivenbycorr(X)= 1.

3 1 ..1 ,( )and the Random variables are said to be Multivariate Normal DistributionDefinition Multivariate Normal Random vectorX=(X1,X2,..,Xn)Tis said to follow a Multivariate Normal distributionwith mean and covariance matrix ifXcanbeexpressedasX=AZ+ ,where =AATandZ=(Z1,Z2,..,Zn) withZi,i=1,2,..,niidN(0,1) 388 BIOS 2083 Linear ModelsAbdus S. WahedBivariate Normal distribution with mean (0,0)Tand covariance matrix 3 2 10123 DensityDefinition Multivariate Normal Random vectorX=(X1,X2,..,Xn)Tis said to follow a Multivariate Normal distributionwith mean and a positive definite covariance matrix ifXhas the densityfX(x)=1(2 )n/2| |1/2exp 12(x )T 1(x ) ( ). Chapter 389 BIOS 2083 Linear ModelsAbdus S. WahedProperties1. Moment generating function of aN( , ) Random variableXis givenbyMX(t)=exp Tt+12tT t .( ) (X)= andcov(X)=.

4 3. IfX1,X2,..,Xnare (0,1) Random variables, then their jointdistribution can be characterized byX=(X1,X2,..,Xn)T N(0,In). Nn( , ) if and only if all non-zero linear combinations of thecomponents ofXare normally 390 BIOS 2083 Linear ModelsAbdus S. WahedLinear transformation5. IfX Nn( , )andAm nis a constant matrix of rankm,thenY=Ax Np(A ,A AT). definition or property 1 linear transformation6. IfX Nn( ,In)andAn nis an orthogonal matrix and =In,thenY=Ax Nn(A ,In). Chapter 391 BIOS 2083 Linear ModelsAbdus S. WahedMarginal and Conditional distributionsSupposeXisNn( , )andXis partitioned as follows,X= X1X2 ,whereX1is of dimensionp 1andX2is of dimensionn p 1. Supposethe corresponding partitions for and are given by = 1 2 ,and = 11 12 21 22 respectively. Then, distribution. X1is Multivariate Normal -Np( 1, 11). the result from property 5 distribution ofX1|X2is p-variate nor-mal -Np( 1|2, 1|2), where, 1|2= 1+ 12 122(X2 2),and 1|2= 11 12 122 21,provided is positive Result , page 156 (Ravishanker and Dey).

5 Chapter 392 BIOS 2083 Linear ModelsAbdus S. WahedUncorrelated implies independence for Multivariate Normal Random vari-ables9. IfX, ,and are partitioned as above, thenX1andX2are independentif and only if 12=0= will use to prove this result. Two Random vectorsX1andX2are independent iffM(X1,X2)(t1,t2)=MX1(t1)MX2(t2).Chapte r 393 BIOS 2083 Linear ModelsAbdus S. Non-central distributionsWe will start with the standard chi-square Chi-square ,X2,..,Xnbeninde-pendentN(0,1) variables, then the distribution of ni=1X2iis 2n(ch-squarewith degrees of freedomn). 2n-distribution is a special case of gamma distribution when the scaleparameter is set to 1/2 and the shape parameter is set to ben/2. That is,the density of 2nis given byf 2n(x)=(1/2)n/2 (n/2)e x/2xn/2 1,x 0;n=1,2,..,.( )Example distribution of (n 1)S2/ 2,whereS2= ni=1(Xi X)2/(n 1) is the sample variance of a Random sample of sizenfrom a normaldistribution with mean and variance 2, follows a 2n moment generating function of a chi-square distribution given byM 2n(t)=(1 2t) n/2,t<1/2.

6 ( )The ( ) shows that the sum of two independent ch-square randomvariables is also a ch-square. Therefore, differences of sequantial sums ofsquares of independent Normal Random variables will be distributed indepen-dently as 394 BIOS 2083 Linear ModelsAbdus S. WahedTheorem Nn( , )and is positive definite, then(X )T 1(X ) 2n.( ) is positive definite, there exists a non-singularAn nsuch that =AAT(Cholesky decomposition). Then, by definition of multivariatenormal distribution,X=AZ+ ,whereZis a Random sample from aN(0,1) distribution. Now, Chapter 395 BIOS 2083 Linear ModelsAbdus S. =0 =2 =4 =6 =8 =10 Figure : Non-central chi-square densities with df 5 and non-centrality parameter .Definition Non-central chi-square sare as in Definition ( ) except that eachXihas mean i,i=1,2,.., , suppose,X=(X1,..,Xn)Tbe a Random vector distributedasNn( ,In), where =( 1.)

7 , n)T. Then the distribution of ni=1X2i=XTXis referred to as non-central chi-square with non-centralityparameter = ni=1 2i/2=12 T . The density of such a non-central chi-square variable 2n( ) can be written as a infinite poisson mixture of centralchi-square densities as follows:f 2n( )(x)= j=1e jj!(1/2)(n+2j)/2 ((n+2j)/2)e x/2x(n+2j)/2 1.( ) Chapter 396 BIOS 2083 Linear ModelsAbdus S. WahedProperties1. The moment generating function of anon-central chi-square variable 2n( )isgivenbyM 2n(n, )(t)=(1 2t) n/2exp 2 t1 2t ,t<1/2.( ) 2n( ) =n+2 . 2n( ) =2(n+4 ).4. 2n(0) For a given constantc,(a)P( 2n( )>c) is an increasing function of .(b)P( 2n( )>c) P( 2n>c). Chapter 397 BIOS 2083 Linear ModelsAbdus S. WahedTheorem Nn( , )and is positive definite, thenXT 1X 2n( = T 1 /2).( ) is positive definite, there exists a non-singular matrixAn nsuch that =AAT(Cholesky decomposition).

8 Define,Y={AT} , Chapter 398 BIOS 2083 Linear ModelsAbdus S. =0 =2 =4 =6 =8 =10 Figure : Non-central F-densities with df 5 and 15 and non-centrality parameter .Definition 2n1( )andU2 2n2andU1andU2are independent, then, the distribution ofF=U1/n1U2/n2( )is referred to as non-centralF-distribution with dfn1andn2, and non-centrality parameter . Chapter 399 BIOS 2083 Linear ModelsAbdus S. Wahed 505101520 =0 =2 =4 =6 =8 =10 Figure : Non-central t-densities with df 5 and non-centrality parameter .Definition N( ,1) andU2 2nandU1andU2are independent, then, the distribution ofT=U1 U2/n( )is referred to as non-centralt-distribution with dfnand non-centrality pa-rameter . Chapter 3100 BIOS 2083 Linear ModelsAbdus S. Distribution of quadratic formsCaution: We assume that our matrix of quadratic form is nis symmetric and idempotent with rankr,thenrofits eigenvalues are exactly equal to1andn rare equal to spectral decomposition theorem.

9 (See Result on page 51 ofRavishanker and Dey).Theorem Nn(0,In). The quadratic formXTAX 2riffAis idempotent withrank(A)= (symmetric) idempotent matrix of rankr. Then, by spectraldecomposition theorem, there exists an orthogonal matrixPsuch thatPTAP= = Ir000 .( )DefineY=PTX= PT1 XPT2X = Y1Y2 ,sothatPT1P1= ,X= Chapter 3101 BIOS 2083 Linear ModelsAbdus S. WahedPYandY1 Nr(0,Ir). Now,XTAx=(PY)TAPY=YT Ir000 Y=YT1Y1 2r.( )Now supposeXTAX 2r. This means that the moment generatingfunction ofXTAXis given byMXTAX(t)=(1 2t) r/2.( )But, one can calculate the ofXTAX directly using the multivariatenormal density asMXTAX(t)=E exp (XTAX)t = exp (XTAX)t fX(x)dx= exp (XTAX)t 1(2 )n/2exp 12xTx dx= 1(2 )n/2exp 12xT(In 2tA)x dx=|In 2tA| 1/2=n i=1(1 2t i) 1/2.( )Equate ( ) and ( ) to obtain the desired 3102 BIOS 2083 Linear ModelsAbdus S. WahedTheorem Nn( , )where is positive definite.

10 The quadraticformXTAX 2r( )where = TA /2,iffA is idempotent withrank(A )= of two quadratic Nn( , )where is positive definite. The two quadratic formsXTAXandXTBXare independent if and only ifA B=0=B A.( ) that in the above theorem, the two quadratic forms neednot have a chi-square distribution. When they are, the theorem is referredto asCraig s of linear and quadratic Nn( , )where is positive definite. The quadratic formXTAX andthe linear formBXare independently distributed if and only ifB A=0.( ) that in the above theorem, the quadratic form need nothave a chi-square 3103 BIOS 2083 Linear ModelsAbdus S. WahedExample Independence of sample mean and sample Nn(0,In). Then X= ni=1Xi/n=1TX/nandS2X= ni=1(Xi X)2/(n 1) are independently 3104 BIOS 2083 Linear ModelsAbdus S. WahedTheorem Nn( , ).ThenE XTAX = TA +trace(A ).( ) that in the above theorem, the quadratic form need nothave a chi-square 3105 BIOS 2083 Linear ModelsAbdus S.