SomeFormulasofMeanandVariance: Weconsidertwo ...

1 Some Formulas of Mean and Variance:We consider tworandom :E(X+Y)=E(X)+E(Y).Proof:For discrete random variablesXandY, it is given by:E(X+Y)= i j(xi+yj)fxy(xi,yj)= i jxifxy(xi,yj)+ i jyjfxy(xi,yj)=E(X)+E(Y).119 For continuous random variablesXandY, we can show:E(X+Y)= (x+y)fxy(x,y)dxdy= xfxy(x,y)dxdy+ yfxy(x,y)dxdy=E(X)+E(Y). :E(XY)=E(X)E(Y), whenXis indepen-dent :For discrete random variablesXandY,E(XY)= i jxiyjfxy(xi,yj)= i jxiyjfx(xi)fy(yj)=( ixifx(xi))( jyjfy(yj))=E(X)E(Y).IfXis independent ofY, the second equality holds, ,fxy(xi,yj)=fx(xi)fy(yj).

2 121 For continuous random variablesXandY,E(XY)= xy fxy(x,y)dxdy= xy fx(x)fy(y)dxdy=( xfx(x)dx)( yfy(y)dy)=E(X)E(Y).WhenXis independent ofY,wehavefxy(x,y)=fx(x)fy(y)in the second :Cov(X,Y)=E(XY) E(X)E(Y).Proof:For both discrete and continuous random variables, wecan rewrite as follows:Cov(X,Y)=E((X x)(Y y))=E(XY xY yX+ x y)=E(XY) E( xY) E( yX)+ x y=E(XY) xE(Y) yE(X)+ x y123=E(XY) x y y x+ x y=E(XY) x y=E(XY) E(X)E(Y).In the fourth equality, the theorem in Section isused, , E( xY)= xE(Y) and E( yX)= yE(X).

3 :Cov(X,Y)=0, whenXis independent :From the above two theorems, we have E(XY)=E(X)E(Y)whenXis independent ofYand Cov(X,Y)=E(XY) E(X)E(Y).Therefore, Cov(X,Y)=0 is obtained whenXis inde-pendent :Thecorrelation coefficient ( )betweenXandY, denoted by xy, is defined as: xy=Cov(X,Y) V(X) V(Y)=Cov(X,Y) x y. xy>0= positive correlationbetweenXandY xy 1= strong positive correlation xy<0= negative correlationbetweenXandY xy 1= strong negative : xy=0, whenXis independent :WhenXis independent ofY,wehaveCov(X,Y)= obtain the result xy=Cov(X,Y) V(X) V(Y)= , note that xy=0 does not mean the indepen-dence :V(X Y)=V(X) 2 Cov(X,Y)+V(Y).

4 Proof:For both discrete and continuous random variables, V(X Y) is rewritten as follows:V(X Y)=E(((X Y) E(X Y))2)=E(((X x) (Y y))2)=E((X x)2 2(X x)(Y y)+(Y y)2)128=E((X x)2) 2E((X x)(Y y))+E((Y y)2)=V(X) 2 Cov(X,Y)+V(Y). : 1 xy :Consider the following function oft:f(t)=V(Xt Y),which is always greater than or equal to zero becauseof the definition of variance. Therefore, for allt,wehavef(t) (t) is rewritten as follows:130f(t)=V(Xt Y)=V(Xt) 2 Cov(Xt,Y)+V(Y)=t2V(X) 2tCov(X,Y)+V(Y)=V(X)(t Cov(X,Y)V(X))2+V(Y) (Cov(X,Y))2V(X).

5 In order to havef(t) 0 for allt, we need the follow-ing condition:V(Y) (Cov(X,Y))2V(X) 0,because the first term in the last equality is nonnega-131tive, which implies:(Cov(X,Y))2V(X)V(Y) , we have: 1 Cov(X,Y) V(X) V(Y) the definition of correlation coefficient, , xy=Cov(X,Y) V(X) V(Y), we obtain the result: 1 xy :V(X Y)=V(X)+V(Y), whenXis inde-pendent :From the theorem above, V(X Y)=V(X) 2 Cov(X,Y)+V(Y) generally holds. When random variablesXandYare independent, we have Cov(X,Y)=0. Therefore,V(X+Y)=V(X)+V(Y) holds, whenXis :Fornrandom variablesX1,X2, ,Xn,E( iaiXi)= iai i,V( iaiXi)= i jaiajCov(Xi,Xj),where E(Xi)= iandaiis a constant value.

6 Espe-cially, whenX1,X2, ,Xnare mutually independent,we have the following:V( iaiXi)= ia2iV(Xi).134 Proof:For mean of iaiXi, the following representation ( iaiXi)= iE(aiXi)= iaiE(Xi)= iai first and second equalities come from the previoustheorems on variance of iaiXi, we can rewrite as follows:V( iaiXi)=E( iai(Xi i))2=E( iai(Xi i))( jaj(Xj j))=E( i jaiaj(Xi i)(Xj j))= i jaiajE((Xi i)(Xj j))= i jaiajCov(Xi,Xj).WhenX1,X2, ,Xnare mutually independent, we136obtain Cov(Xi,Xj)=0 for alli jfrom the previoustheorem.

7 Therefore, we obtain:V( iaiXi)= ia2iV(Xi).Note that Cov(Xi,Xi)=E((Xi )2)=V(Xi). :nrandom variablesX1,X2, ,Xnare mu-tually independently and identically distributed withmean and variance 2. That is, for alli=1,2, ,n,E(Xi)= and V(Xi)= 2are assumed. Considerarithmetic averageX=(1/n) ni=1Xi. Then, mean andvariance ofXare given by:E(X)= ,V(X)= :The mathematical expectation ofXis given by:E(X)=E(1nn i=1Xi)=1nE(n i=1Xi)=1nn i=1E(Xi)=1nn i=1 =1nn = .E(aX)=aE(X) in the second equality and E(X+Y)=E(X)+E(Y) in the third equality are utilized, whereXandYare random variables andais a constant variance ofXis computed as follows.

8 V(X)=V(1nn i=1Xi)=1n2V(n i=1Xi)=1n2n i=1V(Xi)=1n2n i=1 2=1n2n 2= use V(aX)=a2V(X) in the second equality andV(X+Y)=V(X)+V(Y) forXindependent ofYin thethird equality, whereXandYdenote random variablesandais a constant Transformation of Variables ( )Transformation of variables is used in the case of continu-ous random variables. Based on a distribution of a randomvariable, a distribution of the transformed random variable isderived. In other words, when a distribution ofXis known,we can find a distribution ofYusing the transformation ofvariables, whereYis a function Univariate CaseDistribution ofY= 1(X):Letfx(x) be the probabilitydensity function of continuous random variableXandX= (Y) be a one-to-one ( ) transformation.

9 Then, theprobability density function ofY, ,fy(y), is given by:fy(y)=| (y)|fx( (y)).We can derive the above transformation of variables fromXtoYas follows. Letfx(x) andFx(x) be the probability den-142sity function and the distribution function ofX, thatFx(x)=P(X x) andfx(x)=F x(x).WhenX= (Y), we want to obtain the probability densityfunction ofY. Letfy(y) andFy(y) be the probability densityfunction and the distribution function ofY, the case of (X)>0, the distribution function ofY,Fy(y),is rewritten as follows:Fy(y)=P(Y y)=P( (Y) (y))=P(X (y))=Fx( (y)).

10 143 The first equality is the definition of the cumulative distribu-tion function. The second equality holds because of (Y)>0. Therefore, differentiatingFy(y) with respect toy, we canobtain the following expression:fy(y)=F y(y)= (y)F x( (y))= (y)fx( (y)).(4)144 Next, in the case of (X)<0, the distribution function ofY,Fy(y), is rewritten as follows:Fy(y)=P(Y y)=P( (Y) (y))=P(X (y))=1 P(X< (y))=1 Fx( (y)).Thus, in the case of (X)<0, pay attention to the secondequality, where the inequality sign is reversed.