Chapter 4. Multivariate Distributions

1 1 Chapter 4. Multivariate Distributions Joint ( ) Independent Random Variables Covariance and Correlation Coefficient Expectation and Covariance Matrix Multivariate (Normal) Distributions Matlab Codes for Multivariate (Normal) Distributions Some Practical Examples The Joint probability Mass Functions and LetXandYbe two discrete random variables and letRbe the corresponding space ofXandY. The joint ofX=xandY=y, denoted byf(x, y)=P(X=x, Y=y),has the following properties:(a)0 f(x, y) 1for(x, y) R.(b) (x,y) Rf(x, y)=1,(c)P(A)= (x,y) Af(x, y), whereA R.

2 The marginal ofXis defined asfX(x)= yf(x, y),foreachx Rx. The marginal ofYis defined asfY(y)= xf(x, y),foreachy Ry. The random variablesXandYare independent iff (if and only if)f(x, y) fX(x)fY(y)forx Rx,y (x, y)=(x+y)/21,x=1,2,3;y=1,2, thenXandYare (x, y)=(xy2)/30,x=1,2,3;y=1,2, thenXandYare Joint probability Density Functions LetXandYbe two continuous random variables and letRbe the corresponding spaceofXandY. The joint ofX=xandY=y, denoted byf(x, y)=P(X=x, Y=y), has the following properties:(a)f(x, y) 0for <x,y<.

3 (b) f(x, y)dxdy=1.(c)P(A)= Af(x, y), whereA R. The marginal ofXis defined asfX(x)= f(x, y)dy,forx Rx. The marginal ofYis defined asfY(y)= f(x, y)dx,fory Ry. The random variablesXandYare independent iff (if and only if)f(x, y) fX(x)fY(y)forx Rx,y the joint (x, y)=32x2(1 |y|), 1<x<1. 1<y< {(x, y)|0<x<1,0<y<x}.ThenP(A)= 10 x032x2(1 y)dydx= 1032x2[y y22]x0dx= 1032[x3 x42]dx=32[x44 x510]10=940 Example the joint (x, y)=2,0 x y {(x, y)|0 x y 1}.LetA={(x, y)|0 x 12,0 y 12}.ThenP(A)=P(0 X 12,0 Y 12)=P(0 X Y,0 Y 12)= 1/20 y02dxdy=14 Furthermore,fX(x)= 1x2dy=2(1 x),0 x 1and fY(y)= y02dx=2y,0 x Random VariablesThe random variablesXandYare independent iff their joint probability function is theproduct of their marginal distribution functions, that is,f(x, y)=fX(x)fY(y), x, yMore generally, the random variablesX1,X2, ,Xnare mutually independent iff theirjoint probability function is the product of their marginal probability (density) functions, ,f(x1,x2, ,xn)=fX1(x1)fX2(x2)

4 FXn(xn), x1,x2, ,xn(1)LetX1andX2be independent Poisson random variables with respective means 1=2and 2=3. Then(a)P(X1=3,X2=5)=P(X1=3)P(X2=5)=e 2233! e 3355!.(b)P(X1+X2=1)=P(X1=1)P(X2=0)+P(X1= 0)P(X2=1)=e 2211! e 3300!+e 2200! e 3311!.(2)LetX1 b(3, ) andX2 b(5, ) be independent binomial random (a)P(X1=2,X2=4)=P(X1=2)P(X2=4)=(32)( )2(1 )3 2 (54)( )4(1 )5 4(b)P(X1+X2=7)=P(X1=2)P(X2=5)+P(X1=3)P(X 2=4)=(32)( )2(1 )3 2 (55)( )5(1 )5 5+(33)( )3(1 )3 3 (54)( )4(1 )5 4(3)LetX1andX2be two independent randome variables having the same exponentialdistribution with (x)=2e 2x,0<x<.

5 Then(a)E[X1]=E[X2]= [(X1 )2]=E[(X2 )2]= (b)P( <X1< , <X2< ) =( 2xdx) ( 2xdx)(c)E[X1(X2 )2]=E[X1]E[(X2 )2]= = and Correlation CoefficientFor artibrary random variablesXandY, and constantsaandb,wehaveE[aX+bY]=aE[X]+bE[Y ]Proof:We ll show for the continuous case, thediscrete case can be similarly [aX+bY]= (ax+by)f(x, y)dxdy= axf(x, y)dxdy+ byf(x, y)dxdy= ax[ f(x, y)dy]dx+ by[ f(x, y)dx]dy=a xfX(x)dx+b yfY(y)dy=aE[X]+bE[Y]Similarly,E(n i=1aiXi)=n i=1aiE(Xi)Furthermore,E[XY]= xyf(x, y)dxdy[Example]Letf(x, y)=13(x+y),0<x<1,0<y<2, andf(x, y)= [XY]= 10 20xyf(x, y)dydx= 10 20xy13(x+y)dydx=23 LetXandYbe independent random variables, thenE(XY)= xyfX(x)fY(y)dxdy=[ xfX(x)]

6 Dx] [ yfY(y)dy]=E(X) E(Y) The covariance between sXandYis defined asCov(X, Y)=E[(X X)(Y Y)] = (x X)(y Y)f(x, y)dydx=E(XY) X Y IfXandYare independent , thenCov(X, Y)=0. The correlation coefficient is defined by (X, Y)=Cov(X,Y) X Y5 Expectation and Covariance MatrixLetX1,X2,..,Xnbe random variables such that the expectation, variance, and covarianceare defined as follows. j=E(Xj), 2j=Var(Xj)=E[(Xj j)2]Cov(Xi,Xj)=E[(Xi i)(Xj j)] = ij i jSuppose thatX=[X1,X2,..,Xn]tis a random vector, then the expected mean vectorand covariance matrix ofXis defined asE(X)=[ 1, 2.

7 , n]t= Cov(X)=E[(X )(X )t]=[E((Xi i)(Xj j))]Theorem 1:LetX1,X2,..,Xnbenindependent s with respective means{ i}andvariances{ 2i},thenY= ni=1aiXihas mean Y= ni=1ai iand variance 2Y= ni=1a2i 2i, 2:LetX1,X2,..,Xnbenindependent s with respective moment-generatingfunctions{Mi(t)},1 i n, then the moment-generating function ofY= ni=1aiXiisMY(t)= ni=1Mi(ait).6 Multivariate (Normal) Distributions (Gaussian) Normal distribution :X N(u, 2)fX(x)=f(x)=1 2 2exp (x u)2/2 2for <x< mean and variance:E(X)=u, V ar(X)= 2 (Gaussian) Normal distribution :X N(u,C)fX(x)=f(x)=1(2 )d/2[det(C)]1/2e (x u)tC 1(x u)/2forx Rdmean vector and covariance matrix:E(X)=u,Cov(X)=C SimulateX N(u,C)(1)C=LLt,whereLis lower.

8 (2)Generatey N(0,I).(3) x=u+L y(4)Repeat Steps (2)and(3) Simulate N([1 3] , [4,2; 2,5])%n=30;X1=random( normal ,0,1,n,1);X2=random( normal ,0,1,n,1);Y=[ones(n,1), 3*ones(n,1)]+[X1,X2]*[2 1; 0, 2];Yhat=mean(Y) % estimated mean vectorChat=cov(Y) % estimated covariance matrix% Z=[X1, X2];7 Plot a 2D standard Gaussian Distributionx= ;y=x ;X=ones(length(y),1)*x;Y=y*ones(1,length (x));Z=exp(-(X.^2+Y.^2)/2+eps)/(2*pi);me sh(Z);title( f(x,y)= (1/2\pi)*exp[-(x^2+y^2) ] ) (x,y)= (1/2 )*exp[ (x2+y2) ]8 Some Practical Examples(1)LetX1,X2,andX3be independent from a geometric distribution with (x)=(34)(14)x 1,x=1,2, Then(a)P(X1=1,X2=3,X3=1) =P(X1=1)P(X2=3)P(X3=1) =f(1)f(3)f(1)=(34)3(14)2=271024(b)P(X1+X 2+X3=5) = 3P(X1=3,X2=1,X3=1)+3P(X1=2,X2=2,X3=1)=81 512(c) LetY=max{X1,X2,X3},thenP(Y 2) =P(X1 2)P(X2 2)P(X3 2)=(34+34 14)3=(1516)3(2)Let the random variablesXandYhave the joint density functionf(x, y)=xe xy x,x>0,y>0f(x, y)

9 =0elsewhereThen(a)fX(x)= 0xe xy xdx=e x,x>0; X=1, 2X=1.(b)fY(y)=1(1+y)2,y>0; Y= limy [ln(1 +y) 1] does not exist.(c)XandYarenot independentsincef(x, y) =fX(x)fY(y).9(d)P(X+Y 1) = 10( 1 x0xe xy xdy)dx= 10(e x e 2x+x2)dx= 10e xdx e 1 [ 10e1 2x+x2dx]=1 e 1 e 1 ( 10et2dt)(3)Let (X, Y) be uniformly distributed over theunit circle{(x, y):(x2+y2) 1}.Itsjoint is given byf(x, y)=1 ,x2+y2 1f(x, y)=0elsewhere(a)P(X2+Y2 14)= 4 1 .(b){(x, y): (x2+y2) 1,x>y}is a semicircle, soP(X>Y)=12.(c)P(X=Y)=0.(d){(x, y): (x2+y2) 1,x<2y}is a semicircle, soP(Y<2X)=12.

10 (e) LetR=X2+Y2,thenFR(r)=P(R r)=rifr<1, andFR(r)=1ifr 1.(f) ComputefX(x)andfY(y)andshowthatCov(X, Y) = 0 butXandYare ProcessDefinition:A Bernoulli trials process is a sequence of independent and identically dis-tributed (iid) Bernoulli sX1,X2, ,Xn. It is the mathematical model ofnrepetitions of an experiment under identical conditions, with each experiment pro-ducing only two outcomes calledsuccess/failure,head/tail, etc. Two examples aredescribed below.(i) Quality control: As items come off a production line, they are inspected for defects.