The Bivariate Normal Distribution - IIT Kanpur

1 The Bivariate Normal DistributionThis is Section of the 1st edition (2002) of the book Introduc-tion to Probability, by D. P. Bertsekas and J. N. Tsitsiklis. Thematerial in this section was not included in the 2nd edition (2008).LetUandVbe two independent Normal random variables, and consider twonew random variablesXandYof the formX=aU+bV,Y=cU+dV,wherea, b, c, d, are some scalars. Each one of the random variablesXandYisnormal, since it is a linear function of independent Normal random variables. Furthermore, becauseXandYare linear functions of the same two independentnormal random variables, their joint PDF takes a special form, known as thebi-variate normalPDF.

2 The Bivariate Normal PDF has several useful and elegantproperties and, for this reason, it is a commonly employed model. In this section,we derive many such properties, both qualitative and analytical, culminating ina closed-form expression for the joint PDF. To keep the discussion simple, werestrict ourselves to the case whereXandYhave zero Normal Random VariablesTwo random variablesXandYare said to bejointly normalif they canbeexpressedintheformX=aU+bV,Y=cU+dV,w hereUandVare independent Normal random that ifXandYare jointly Normal , then any linear combinationZ=s1X+s2Y For the purposes of this section, we adopt the following convention.

3 A randomvariable which is always equal to a constant will also be called Normal , with zerovariance, even though it does not have a PDF. With this convention, the family ofnormal random variables is closed under linear operations. That is, ifXis Normal ,thenaX+bis also Normal , even ifa= Bivariate Normal Distributionhas a Normal Distribution . The reason is that if we haveX=aU+bVandY=cU+dVfor some independent Normal random variablesUandV,thenZ=s1(aU+bV)+s2(cU+dV) =(as1+cs2)U+(bs1+ds2) ,Zis the sum of the independent Normal random variables (as1+cs2)Uand (bs1+ds2)

4 V, and is therefore very important property of jointly Normal random variables, and whichwill be the starting point for our development, is that zero correlation Correlation Implies IndependenceIf two random variablesXandYare jointly Normal and are uncorrelated,then they are property can be verified using multivariate transforms, as thatUandVare independent zero-mean Normal random variables,and thatX=aU+bVandY=cU+dV,sothatXandYare jointly assume thatXandYare uncorrelated, and we wish to show that they areindependent. Our first step is to derive a formula for the multivariate transformMX,Y(s1,s2) associated withXandY.

5 Recall that ifZis a zero-mean normalrandom variable with variance 2Z, the associated transform isE[esZ]=MZ(s)=e 2Zs2/2,which implies thatE[eZ]=MZ(1) =e 2 us fix some scalarss1,s2,andletZ=s1X+s2Y. The random variableZis Normal , by our earlier discussion, with variance 2Z=s21 2X+s22 leads to the following formula for the multivariate transform associatedwith the uncorrelated pairXandY:MX,Y(s1,s2)=E[es1X+s2Y]=E[eZ]= e(s21 2X+s22 2Y) nowXandYbeindependentzero-mean Normal random variables withthe same variances 2 Xand 2 YasXandY, respectively.

6 SinceXandYareindependent, they are also uncorrelated, and the preceding argument yieldsMX,Y(s1,s2)=e(s21 2X+s22 2Y) Bivariate Normal Distribution3 Thus, the two pairs of random variables (X, Y)and(X,Y) are associated withthe same multivariate transform. Since the multivariate transform completelydetermines the joint PDF, it follows that the pair (X, Y) has the same jointPDF as the pair (X,Y). SinceXandYare independent,XandYmust alsobe independent, which establishes our Conditional Distribution ofXGivenYWe now turn to the problem of estimatingXgiven the value degenerate cases, we assume that bothXandYhave positivevariance.

7 Let us define X= X YY, X=X X,where =E[XY] X Yis the correlation coefficient linear combinationsof independent Normal random variablesUandV, it follows thatYand Xarealso linear combinations ,Yand Xare jointly ,E[Y X]=E[YX] E[Y X]= X Y X Y 2Y= ,Yand Xare uncorrelated and, therefore, independent. Since Xis a scalarmultiple ofY, it follows that Xand Xare have so far decomposedXinto a sum of two independent Normal ran-dom variables, namely,X= X+ X= X YY+ take conditional expectations of both sides, givenY,toobtainE[X|Y]= X YE[Y|Y]+E[ X|Y]= X YY= X,where we have made use of the independence ofYand Xto setE[ X|Y]=0.

8 Wehave therefore reached the important conclusion that the conditional expectationE[X|Y] is a linear function of the random the above decomposition, it is now easy to determine the conditionalPDF ofX. Given a value ofY, the random variable X= XY/ Ybecomes Comparing with the formulas in the preceding section, it is seen that Xisdefined to be the linear least squares estimator ofX,and Xis the correspondingestimation error, although these facts are not needed for the argument that Bivariate Normal Distributiona known constant, but the Normal Distribution of the random variable Xisunaffected, since Xis independent ofY.

9 Therefore, the conditional distributionofXgivenYis the same as the unconditional Distribution of X,shiftedby Xis Normal with mean zero and some variance 2 X, we conclude that theconditional Distribution ofXis also Normal with mean Xand the same variance 2 X. The variance of Xcan be found with the following calculation: 2 X=E[(X X YY)2]= 2X 2 X Y X Y+ 2 2X 2Y 2Y=(1 2) 2X,wherewehavemadeuseofthepropertyE[XY]= X summarize our conclusions below. Although our discussion used thezero-mean assumption, these conclusions also hold for the non-zero mean caseand we state them with this added generality; see the end-of-chapter of Jointly Normal Random VariablesLetXandYbe jointly Normal random variables.

10 XandYare independent if and only if they are uncorrelated. The conditional expectation ofXgivenYsatisfiesE[X|Y]=E[X]+ X Y(Y E[Y]).It is a linear function ofYand has a Normal PDF. The estimation error X=X E[X|Y] is zero-mean, Normal , andindependent ofY, with variance 2 X=(1 2) 2X. The conditional Distribution ofXgivenYis Normal with meanE[X|Y]and variance 2 determined the parameters of the PDF of Xand of the conditional PDFofX, we can give explicit formulas for these PDFs. We keep assuming thatThe Bivariate Normal Distribution5 XandYhave zero means and positive variances.