[Chapter 5. Multivariate Probability Distributions]

1 [ chapter 5. MultivariateProbability Distributions] and Multivariate Probability and Conditional Probability random expected value of a function of ran-dom Covariance of two random Moments of linear combinations ofrandom Multinomial Probability Bivariate normal IntroductionSuppose thatY1,Y2,..,Yndenote the outcomesofnsuccessive trials of an set of outcomes, or sample measure-ments, may be expressed in terms of the inter-section ofnevents(Y1=y1),(Y2=y2),..,(Yn=yn)which we will denote as(Y1=y1,Y2=y2,..,Yn=yn)or more compactly, as(y1,y2,..,yn).Calculation of the Probability of this intersec-tion is essential in making inferences about thepopulation from which the sample was drawnand is a major reason for studying multivariateprobability Bivariate and Multivariate probabil-ity distributionsMany random variables can be defined over thesame sample space.

2 (Example) Tossing a pair of sample space contains 36 sample the number of dots appearing ondie 1, andY2be the sum of the number ofdots on the dice. We would like to obtain theprobability of (Y1=y1,Y2=y2) for all thepossible values ofy1andy2. That is the jointdistribution ofY1andY2.(Def ) For any joint (bi-variate) distribution functionF(y1,y2) is givenbyF(y1,y2) =P(Y1 y1,Y2 y2)for < y1< and < y2< .3(Theorem )IfY1andY2are with joint dis-tribution functionF(y1,y2), ( , ) =F( ,y2) =F(y1, ) = ( , ) = Ifa?1 a1andb?2 b2, thenF(a?1,b?2) F(a?1,b2) F(a1,b?2) +F(a1,b2)=P(a1< Y1 a?1,b2< Y2 b?2) 0.(1) Discrete variables:(Def ) LetY1andY2be discrete Thejoint Probability distributionforY1andY2isgiven byp(y1,y2) =p(Y1=y1,Y2=y2)for < y1< and < y2< . Thefunctionp(y1,y2) will be referred to as the jointprobability that ifY1andY2are discrete withjoint Probability functionp(y1,y2), its CDF isF(y1,y2) =P(Y1 y1,Y2 y2)= t1 y1 t2 y2p(t1,t2)(Theorem ) IfY1andY2are discrete joint Probability functionp(y1,y2), (y1,y2) 0 for ally1, y1,y2p(y1,y2) = 1, where the sum is overall values (y1,y2) that are assigned [(y1,y2) A]= (y1,y2) Ap(y1,y2) forA S.

3 So,P(a1 Y1 a2,b1 Y2 b2) =a2 t1=a1b2 t2=b1p(t1,t2)5(Example ) A local supermarket has threecheckout counters. Two customers arrive atthe counters at different times when the coun-ters are serving no other customers. Each cus-tomer chooses a counter at random, indepen-dently of the other. LetY1denote the numberof customers who choose counter 1 andY2,the number who select counter 2. Find thejoint distribution ofY1andY2.(Example ) Consider theY1andY2in (Ex-ample ). FindF( 1,2),F( ,2) andF(5,7).6(2) Continuous variables:Two random variables are said to be jointlycontinuous if their joint distribution functionF(y1,y2) is continuous in both arguments.(Def ) LetY1andY2be continuous joint distribution functionF(y1,y2).Ifthere exists a nonnegative functionf(y1,y2)such thatF(y1,y2) = y1 y2 f(t1,t2)dt2dt1for all < y1< and < y2< , thenY1andY2are said to be jointly continuousrandom variables.

4 The functionf(y1,y2) willbe referred to as the joint Probability (Theorem ) IfY1andY2are jointly con-tinuous random variables with a joint densityfunctionf(y1,y2), (y1,y2) 0 for ally1, f(y1,y2)dy1dy2= [(y1,y2) A]= Af(y1,y2)dy2dy1. So,P(a1 Y1 a2,b1 Y2 b2) = b2b1 a2a1f(y1,y2)dy1dy2(Example )(Example )(Exercise )(Exercise )8(Question) How about the case of the inter-section ofnevents(Y1=y1,Y2=y2,..,Yn=yn)?For discrete ,the Probability function is given byp(y1,y2,..,yn) =P(Y1=y1,Y2=y2,..,Yn=yn)and its joint distribution function is given byF(y1,y2,..,yn) =P(Y1 y1,Y2 y2,..,Yn yn)= t1 y1 t2 y2 tn ynp(y1,y2,..,yn).For continuous ,the joint distribution function is given byP(Y1 y1,Y2 y2,..,Yn yn) =F(y1,..,yn)= y1 y2 .. yn f(t1,t2,..,tn) every set of real numbers (y1,y2,..,yn) andits joint density is given byf(y1,y2.)

5 ,yn). Marginal and Conditional probabil-ity distributionsGiven Theorem and Definition ,[Discrete random variables](Def ) jointly discrete withprobability functionp(y1,y2). Then themarginalprobability functionsofY1andY2are given byp1(y1) = y2p(y1,y2), p2(y2) = y1p(y1,y2).(Def )IfY1andY2are jointly discrete with jointprobability functionp(y1,y2) and marginal prob-ability functionsp1(y1) andp2(y2) respectively,then the conditional discrete Probability func-tion ofY1givenY2isp(y1|y2) =P(Y1=y1|Y2=y2)=P(Y1=y1,Y2=y2)P(Y2=y2)=p (y1,y2)p2(y2)provided thatp2(y2)> that1. Multiplicative law(Theorem ( )):P(A B) =P(A)P(B|A).2. Consider the intersection of the two nu-merical events, (Y1=y1) and (Y2=y2),represented by the bivariate event (y1,y2).Then, the bivariate Probability for (y1,y2)isp(y1,y2) =p1(y1)p(y2|y1) =p2(y2)p(y1|y2) (y1|y2) : the Probability that the , given that thatY2takes on (y1|y2) is undefined ifp2(y2) = 0.

6 (Example , )11(Example) Contracts for two construction jobsare randomly assigned to one or more of threefirmsA,BandC. LetY1andY2be the numberof contracts assigned to firmAandB, respec-tively. Recall that each firm can receives 0, 1,or 2 Find the joint Probability distribution CalculateF(1,0),F(3,4) andF( , )c. Find the marginal Probability distribution Find the conditional Probability function forY2givenY1= Find the conditional Probability function forY2givenY1= Definition and Theorem ,[Continuous random variables](Def ) jointly continuous Probability functionf(y1,y2). Then themarginal density functionsofY1andY2aregiven byf1(y1) = f(y1,y2)dy2, f2(y2) = f(y1,y2) continuousY1andY2,P(Y1=y1|Y2=y2) can not be defined as in the discrete case,because both (Y1=y1) and (Y2=y2) areevents with zero Probability .(Def )IfY1andY2are jointly continuous withjoint density functionf(y1,y2), then the condi-tional distribution function ofY1givenY2=y2isF(y1|y2) =P(Y1 y1|Y2=y2) = y1 f(t1,y2)f2(y2)dt113 Note that one can derive conditional densityfunction ofY1givenY2=y2,f(y1|y2) fromthe calculation ofF(y1) :(Def )IfY1andY2are jointly continuous withjoint density functionf(y1,y2) and marginaldensitiesf1(y1) andf2(y2), respectively.

7 Foranyy2such thatf2(y2)>0, the conditionaldensity ofY1givenY2=y2is given byf(y1|y2) =f(y1,y2)f2(y2).and, for anyy1such thatf1(y1)>0, the con-ditional density ofY2givenY1=y1is givenbyf(y2|y1) =f(y1,y2)f1(y1).Note that i)f(y1|y2) is undefined for ally2such thatf2(y2) = 0, ii)f(y2|y1) is undefinedfor ally1such thatf1(y1) = 0.(Example )14(Example) LetY1andY2have joint probabilitydensity function(pdf ) given byf(y1,y2) =k(1 y2)0 y1 y2 1= 0elsewherea. Find the value of k such that this is a CalculateP(Y1 3/4,Y2 1/2)c. Find the marginal density function CalculateP(Y1 1/2|Y2 3/4)e. Find the conditional density function Find the conditional density function CalculateP(Y2 3/4|Y1= 1/2) Independent random variables Independent random variables :Two eventsAandBare independent ifP(A B) =P(A)P(B).Suppose we are concerned with events ofthe type (a Y1 b) (c Y2 d).

8 IfY1andY2are independent, does the followingequation hold?P(a Y1 b,c Y2 d) =P(a Y1 b)P(c Y2 d)(Def )LetY1have distribution functionF1(y1),Y2have distribution functionF2(y2), andY1andY2have joint distribution functionF(y1,y2).Then,Y1andY2are said to beindependentif and only ifF(y1,y2) =F1(y1)F2(y2)for every pair of real numbers (y1,y2). IfY1andY2are not independent, they are said of (Def ) tondimensions:Suppose we havenrandom variables,Y1,..,Yn,whereYihas distribution functionFi(yi), fori= 1,2,..,n; and whereY1,..,Ynhave jointdistributionF(y1,y2,..,yn).ThenY1,. .,Ynare independent if and only ifF(y1,y2,..,yn) =F1(y1) Fn(yn)for all real numbersy1,y2,..,yn.(Theorem ) Discrete : IfY1andY2are joint Probability functionp(y1,y2)and marginal Probability functionsp1(y1) andp2(y2) respectively, thenY1andY2are inde-pendent if and only ifp(y1,y2) =p1(y1)p2(y2)for all pair of real numbers (y1,y2).

9 17(Theorem ) Continuous : IfY1andY2are with joint density functionf(y1,y2) andmarginal density functionsf1(y1) andf2(y2)respectively, thenY1andY2are independent ifand only iff(y1,y2) =f1(y1)f2(y2)for all pair of real numbers (y1,y2).(Example )(Example )18 The key benefit of the following theorem isthat we do not actually need to derive themarginal densities. Indeed, the functionsg(y1)andh(y2) need not, themselves, be densityfunctions.(Theorem ) IfY1andY2have a joint densityf(y1,y2) that is positive if and only ifa y1 bandc y2 d, for constantsa,b,c,andd; andf(y1,y2) = 0 otherwise. ThenY1andY2areindependent if and only iff(y1,y2) =g(y1)h(y2)whereg(y1) is a nonnegative function ofy1alone andh(y2) is a nonnegative function ofy2alone.(Example )(Exercise ) Expected value of a function of (Def ) Discrete : Letg(Y1,Y2.)

10 ,Yk) a functionof the discrete ,Y1,Y2,..,Yk, which haveprobability functionp(y1,y2,..,yk). Then theexpected valueofg(Y1,Y2,..,Yk) isE[g(Y1,Y2,..,Yk)]= yk y2 y1g(y1,y2,..,yk)p(y1,y2,..,yk) Continuous : IfY1,..,Ykare with joint density functionf(y1,..,yk),thenE[g(Y1,..,Yk)]= yk y1g(y1,..,yk)f(y1,..,yk) Derivation ofE(Y1) from (Def )(and ).(Example )(Example ) Special theorems(Theorem )Letcbe a constant. ThenE(c) =c(Theorem ) Letg(Y1,Y2) be a function ofthe , and letcbe a [cg(Y1,Y2)] =cE[g(Y1,Y2)](Theorem )LetY1,Y2be andg1(Y1,Y2),g2(Y1,Y2),..,gk(Y1,Y2) be functions ofY1andY2. ThenE[g1(Y1,Y2) +g2(Y1,Y2) +..+gk(Y1,Y2)]=E[g1(Y1,Y2)] +E[g2(Y1,Y2)] + +E[gk(Y1,Y2)].(Example )21(Theorem ) LetY1andY2be andg(Y1) andh(Y2) be functions of onlyY1andY2, respectively. ThenE[g(Y1)h(Y2)] =E[g(Y1)]E[h(Y2)].