General Bivariate Normal - Duke University

1 Lecture 22: Bivariate Normal DistributionStatistics 104 Colin RundelApril 11, DistributionsGeneral Bivariate NormalLetZ1,Z2 N(0,1), which we will use to build a General Bivariate (z1,z2) =12 exp[ 12(z21+z22)]We want to transform these unit Normal distributions to have the followarbitrary parameters: X, Y, X, Y, X= XZ1+ XY= Y[ Z1+ 1 2Z2] + YStatistics 104 (Colin Rundel)Lecture 22 April 11, 20121 / DistributionsGeneral Bivariate Normal - MarginalsFirst, lets examine the marginal distributions ofXandY,X= XZ1+ X= XN(0,1) + X=N( X, 2X)Y= Y[ Z1+ 1 2Z2] + Y= Y[ N(0,1) + 1 2N(0,1)] + Y= Y[N(0, 2) +N(0,1 2)] + Y= YN(0,1) + Y=N( Y, 2Y)Statistics 104 (Colin Rundel)Lecture 22 April 11, 20122 / DistributionsGeneral Bivariate Normal - Cov/CorrSecond, we can findCov(X,Y) and (X,Y)Cov(X,Y) =E[(X E(X))(Y E(Y))]=E[( XZ1+ X X)( Y[ Z1+ 1 2Z2] + Y Y)]=E[( XZ1)( Y[ Z1+ 1 2Z2])]= X YE[ Z21+ 1 2Z1Z2]= X Y E[Z21]= X Y (X,Y)

2 =Cov(X,Y) X Y= Statistics 104 (Colin Rundel)Lecture 22 April 11, 20123 / DistributionsGeneral Bivariate Normal - RNGC onsequently, if we want to generate a Bivariate Normal random variablewithX N( X, 2X) andY N( Y, 2Y) where the correlation ofXandYis we can generate two independent unit normalsZ1andZ2and usethe transformation:X= XZ1+ XY= Y[ Z1+ 1 2Z2] + YWe can also use this result to find the joint density of the BivariateNormal using a 2d change of 104 (Colin Rundel)Lecture 22 April 11, 20124 / DistributionsMultivariate Change of VariablesLetX1,..,Xnhave a continuous joint distribution with pdffdefined ofS. We can definennew random variablesY1.

3 ,Ynas follows:Y1=r1(X1,..,Xn) Yn=rn(X1,..,Xn)If we assume that thenfunctionsr1,..,rndefine a one-to-one differentiable transformationfromStoTthen let the inverse of this transformation bex1=s1(y1,..,yn) xn=sn(y1,..,yn)Then the joint pdfgofY1,..,Ynisg(y1,..,yn) ={f(s1,..,sn)|J|for (y1,..,yn) T0otherwiseWhereJ= det s1 y1 s1 sn y1 sn yn Statistics 104 (Colin Rundel)Lecture 22 April 11, 20125 / DistributionsGeneral Bivariate Normal - DensityThe first thing we need to find are the inverses of the transformation. Ifx=r1(z1,z2) andy=r2(z1,z2) we need to find functionsh1andh2suchthatZ1=s1(X,Y) andZ2=s2(X,Y).X= XZ1+ XZ1=X X XY= Y[ Z1+ 1 2Z2] + YY Y Y= X X X+ 1 2Z2Z2=1 1 2[Y Y Y X X X]Therefore,s1(x,y) =x X Xs2(x,y) =1 1 2[y Y Y x X X]Statistics 104 (Colin Rundel)Lecture 22 April 11, 20126 / DistributionsGeneral Bivariate Normal - DensityNext we calculate the Jacobian,J= det[ s1 x s1 y s2 x s2 y]= det[1 X0 X 1 21 Y 1 2]=1 X Y 1 2 The joint density ofXandYis then given byf(x,y) =f(z1,z2)|J|=12 exp[ 12(z21+z22)]|J|=12 X Y 1 2exp[ 12(z21+z22)]=12 X Y 1 2exp[ 12[(x X X)2+11 2(y Y Y x X X)2]]=12 X Y(1 2)1/2exp[ 12(1 2)((x X)2 2X+(y Y)2 2Y 2 (x X) X(y Y) Y)]Statistics 104 (Colin Rundel)}

4 Lecture 22 April 11, 20127 / DistributionsGeneral Bivariate Normal - Density (Matrix Notation)Obviously, the density for the Bivariate Normal is ugly, and it only getsworse when we consider higher dimensional joint densities of normals. Wecan write the density in a more compact form using matrix notation,x=(xy) =( X Y) =( 2X X Y X Y 2Y)f(x) =12 (det ) 1/2exp[ 12(x )T 1(x )]We can confirm our results by checking the value of (det ) 1/2and(x )T 1(x ) for the Bivariate case.(det ) 1/2=( 2X 2Y 2 2X 2Y) 1/2=1 X Y(1 2)1/2 Statistics 104 (Colin Rundel)Lecture 22 April 11, 20128 / DistributionsGeneral Bivariate Normal - Density (Matrix Notation)Recall for a 2 2 matrix,A=(abcd)A 1=1detA(d b ca)=1ad bc(d b ca)Then,(x )T 1(x )=1 2X 2Y(1 2)(x xy y)T( 2Y X Y X Y 2X)(x xy y)=1 2X 2Y(1 2)( 2Y(x X) X Y(y Y) X Y(x X) + 2X(y Y))T(x xy y)=1 2X 2Y(1 2)( 2Y(x X)2 2 X Y(x X)(y Y) + 2X(y Y)2)=11 2((x X)2 2X 2 (x X)(y Y) X Y+(y Y)2 2Y)Statistics 104 (Colin Rundel)Lecture 22 April 11, 20129 / DistributionsGeneral Bivariate Normal - ExamplesX N(0,1),Y N(0,1) = 0X N(0,2)

5 ,Y N(0,1) = 0X N(0,1),Y N(0,2) = 0 Statistics 104 (Colin Rundel)Lecture 22 April 11, 201210 / DistributionsGeneral Bivariate Normal - ExamplesX N(0,1),Y N(0,1) = N(0,1),Y N(0,1) = N(0,1),Y N(0,1) = 104 (Colin Rundel)Lecture 22 April 11, 201211 / DistributionsGeneral Bivariate Normal - ExamplesX N(0,1),Y N(0,1) = N(0,1),Y N(0,1) = N(0,1),Y N(0,1) = 104 (Colin Rundel)Lecture 22 April 11, 201212 / DistributionsGeneral Bivariate Normal - ExamplesX N(0,1),Y N(0,1) = N(0,2),Y N(0,1) = N(0,1),Y N(0,2) = 104 (Colin Rundel)Lecture 22 April 11, 201213 / DistributionsMultivariate Normal DistributionMatrix notation allows us to easily express the density of the multivariatenormal distribution for an arbitrary number of dimensions.

6 We express thek-dimensional multivariate Normal distribution as follows,X Nk( , )where is thek 1 column vector of means and is thek kcovariance matrix where{ }i,j=Cov(Xi,Xj).The density of the distribution isf(x) =1(2 )k/2(det ) 1/2exp[ 12(x )T 1(x )]Statistics 104 (Colin Rundel)Lecture 22 April 11, 201214 / DistributionsMultivariate Normal distribution - CholeskyIn the Bivariate case, we had a nice transformation such that we couldgenerate two independent unit Normal values and transform them into asample from an arbitrary Bivariate Normal is a similar method for the multivariate Normal distribution thattakes advantage of the Cholesky decomposition of the covariance Cholesky decomposition is defined for a symmetric, positive definitematrixXasL= Chol(X)

7 WhereLis a lower triangular matrix such thatLLT= 104 (Colin Rundel)Lecture 22 April 11, 201215 / DistributionsMultivariate Normal distribution - RNGLetZ1,..,Zk N(0,1) andZ= (Z1,..,Zk)Tthen + Chol( )Z Nk( , )this is offered without proof in the generalk-dimensional case but we cancheck that this results in the same transformation we started with in thebivariate case and should justify how we knew to use that 104 (Colin Rundel)Lecture 22 April 11, 201216 / DistributionsCholesky and the Bivariate TransformationWe need to find the Cholesky decomposition of for the General bivariatecase where =( 2X X Y X Y 2Y)We need to solve the following fora,b,c(a0bc)(ab0c)=(a2ababb2+c2)=( 2X X Y X Y 2Y)This gives us three (unique)

8 Equations and three unknowns to solve for,a2= 2 Xab= X Yb2+c2= 2Ya= Xb= X Y/a= Yc= 2Y b2= Y(1 2)1/2 Statistics 104 (Colin Rundel)Lecture 22 April 11, 201217 / DistributionsCholesky and the Bivariate TransformationLetZ1,Z2 N(0,1) then(XY)= + Chol( )Z=( X Y)+( X0 Y Y(1 2)1/2)(Z1Z2)=( X Y)+( XZ1 YZ1+ Y(1 2)1/2Z2)X= X+ XZ1Y= Y+ Y[ Z1+ (1 2)1/2Z2]Statistics 104 (Colin Rundel)Lecture 22 April 11, 201218 / DistributionsConditional Expectation of the Bivariate NormalUsingX= X+ XZ1andY= Y+ Y[ Z1+ (1 2)1/2Z2] whereZ1,Z2 N(0,1) we can findE(Y|X).E[Y|X=x] =E[ Y+ Y( Z1+ (1 2)1/2Z2) X=x]=E[ Y+ Y( x X X+ (1 2)1/2Z2) X=x]= Y+ Y( x X X+ (1 2)1/2E[Z2|X=x])= Y+ Y (x X X)By symmetry,E[X|Y=y] = X+ X (y Y Y)Statistics 104 (Colin Rundel)Lecture 22 April 11, 201219 / DistributionsConditional Variance of the Bivariate NormalUsingX= X+ XZ1andY= Y+ Y[ Z1+ (1 2)1/2Z2] whereZ1,Z2 N(0,1) we can findVar(Y|X).

9 Var[Y|X=x] =Var[ Y+ Y( Z1+ (1 2)1/2Z2) X=x]=Var[ Y+ Y( x X X+ (1 2)1/2Z2) X=x]=Var[ Y(1 2)Z2|X=x]= 2Y(1 2)By symmetry,Var[X|Y=y] = 2X(1 2)Statistics 104 (Colin Rundel)Lecture 22 April 11, 201220 / DistributionsExample - Husbands and Wives(Example , deGroot)Suppose that the heights of married couples can be explained by a Bivariate Normal the wives have a mean heigh of inches and a standard deviation of 2 inches while theheights of the husbands have a mean of 70 inches and a standard deviation of 2 inches. Thecorrelation between the heights is What is the probability that for a randomly selectedcouple the wife is taller than her husband?

10 Statistics 104 (Colin Rundel)Lecture 22 April 11, 201221 / DistributionsExample - ConditionalsSuppose thatX1andX2have a Bivariate Normal distribution whereE(X1|X2) = ,E(X2|X1) = , andVar(X2|X1) = (X1),Var(X1),E(X2),Var(X2), and (X1,X2).Statistics 104 (Colin Rundel)Lecture 22 April 11, 201222 / DistributionsExample - Conditionals, 104 (Colin Rundel)Lecture 22 April 11, 201223 / 22