Expected Value, Variance and Covariance

1 Expected ValueVarianceCovarianceExpected Value, Variance and Covariance (Sections )1 STA 256: Fall 20191 This slide show is an open-source document. See last slide for / 31 Expected ValueVarianceCovarianceOverview1 Expected Value2 Variance3 Covariance2 / 31 Expected ValueVarianceCovarianceDefinition for Discrete Random VariablesThe Expected value of a discrete random variable isE(X) = xxpX(x)Provided x|x|pX(x)< .If the sum diverges, theexpected value does not is only an issue for infinite sums (and integralsover infinite intervals).3 / 31 Expected ValueVarianceCovarianceExpected value is an averageImagine a very large jar full of is the balls are numberedx1,.., aremeasurements carried out on members of the for now that all the numbers are ball is selected at random; all balls are equally likely tobe the number on the ball (X=xi) = (X) = xxpX(x)=N i=1xi1N= Ni=1xiN4 / 31 Expected ValueVarianceCovarianceFor the jar full of numbered balls,E(X) = Ni=1xiNThis is the common average, or arithmetic there are values arevi, fori= 1.

2 , have valuev1,andn2balls have valuev2, and ..nnballshave + +nn=N, andP(X=vj) = (X) = Ni=1xiN=n j=1njvj1N=n j=1vjnjN=n j=1vjP(X=vj)5 / 31 Expected ValueVarianceCovarianceCompareE(X) = nj=1vjP(X=vj) and xxpX(x) Expected value is a generalization of the idea of an average,or is often just called the mean. 6 / 31 Expected ValueVarianceCovarianceGambling InterpretationPlay a game for be a casino game, or a business game like placing abid on a the return that is, be the game over and over (independently).The long term average return isE(X).This follows from the Law of Large Numbers, a theoremthat will be proved / 31 Expected ValueVarianceCovarianceFair GameDefinition:A fair game is one withexpected value equal to / 31 Expected ValueVarianceCovarianceRational BehaviourMaximize Expected return (it does not have to be money)At least, don t play any games with a negative / 31 Expected ValueVarianceCovarianceExamplePlace a $20 bet, roll a fair it s a 6, you get your $20 back and an additional $ it s not a 6, you lose your $ this a fair game?

3 E(X) = ( 20)56+ (100)16=16( 100 + 100)= / 31 Expected ValueVarianceCovarianceDefinition for Continuous Random VariablesThe Expected value of a continuous random variable isE(X) = xfX(x)dxProvided |x|fX(x)dx < .If the integral diverges,the Expected value does not / 31 Expected ValueVarianceCovarianceThe Expected value is the physical / 31 Expected ValueVarianceCovarianceSometimes the Expected value does not existNeed |x|fX(x)dx < For the Cauchy distribution,f(x) =1 (1+x2).E(|X|) = |x|1 (1 +x2)dx= 2 0x (1 +x2)dxu= 1 +x2, du= 2xdx=1 11udu=1 lnu| 1= 0= So to we say an integral equals infinity, we justmean it is unbounded / 31 Expected ValueVarianceCovarianceExistence of Expected valuesIf it is not mentioned in a general problem, existence ofexpected values is , the answer to a specific problem is Oops! Theexpected value dies not exist. You never need to show existence unless you are explicitlyasked to do you do need to deal with existence, Fubini s Theoremcan help with multiple sums or One says that if the integrand is positive, the answer isthe same when you switch order of integration, even whenthe answer is.

4 Part Two says that if the integral converges absolutely, youcan switch order of us, absoluteconvergence just means that the Expected value / 31 Expected ValueVarianceCovarianceThe change of variables formula for Expected valueTheorems and a random variable andY=g(X). There are two waysto getE(Y).1 Derive the distribution ofYand computeE(Y) = y fY(y)dy2 Use the distribution ofXand calculateE(g(X)) = g(x)fX(x)dxBig theorem: These two expressions are / 31 Expected ValueVarianceCovarianceThe change of variables formula is very generalIncluding but not limited toE(g(X)) = g(x)fX(x)dxE(g(X)) = g(x1,..,xp)fX(x1,..,xp) (g(X)) = xg(x)pX(x)E(g(X)) = x1 xpg(x1,..,xp)pX(x1,..,xp)16 / 31 Expected ValueVarianceCovarianceExample: LetY=aX. FindE(Y).E(aX) = xaxpX(x)=a xxpX(x)=aE(X)SoE(aX) =aE(X).17 / 31 Expected ValueVarianceCovarianceShow that the Expected value of a constant is (a) = xapX(x)=a xpX(x)=a 1=aSoE(a) = / 31 Expected ValueVarianceCovarianceE(X+Y) =E(X) +E(Y)E(X+Y) = (x+y)fX,Y(x,y)dxdy= xfX,Y(x,y)dxdy+ y fX,Y(x,y)dxdy=E(X) +E(Y)19 / 31 Expected ValueVarianceCovariancePutting it togetherE(a+bX+cY) =a+bE(X) +cE(Y)And in fact,E(n i=1aiXi)=n i=1aiE(Xi)You can move the Expected value sign through summation signsand value is a linear / 31 Expected ValueVarianceCovarianceE( ni=1Xi) = ni=1E(Xi), but in generalE(g(X))6=g(E(X))Unlessg(x) is a linear for example,E(ln(X))6= ln(E(X))E(1X)6=1E(X)E(Xk)6= (E(X))kThat is, the statements are not true in general.

5 They might betrue for some / 31 Expected ValueVarianceCovarianceVariance of a random variableXLetE(X) = (The Greek letter mu ).V ar(X) =E((X )2)The average (squared) difference from the s a measure of how spread out the distribution measure of spread is the standard deviation, thesquare root of the / 31 Expected ValueVarianceCovarianceVariance rulesV ar(a+bX) =b2V ar(X)V ar(X) =E(X2) [E(X)]223 / 31 Expected ValueVarianceCovarianceConditional ExpectationThe ideaConsider jointly distributed random each possible value ofX, there is a conditionaldistribution conditional distribution has an Expected value(sub-population mean).If you could estimateE(Y|X=x), it would be a good wayto comes later (in STA260).24 / 31 Expected ValueVarianceCovarianceDefinition of Conditional ExpectationIfXandYare discrete, the conditional Expected value ofYgivenXisE(Y|X=x) = yy pY|X(y|x)IfXandYare continuous,E(Y|X=x) = y fY|X(y|x)dy25 / 31 Expected ValueVarianceCovarianceDouble Expectation:E(Y) =E[E(Y|X)]Theorem A on page 149To make sense of this, noteWhileE(Y|X=x) = y fY|X(y|x)dyis a real-valuedfunction ofx,E(Y|X) is a random variable, a function of the (Y|X) =g(X) = y fY|X(y|X) that inE[E(Y|X)] =E[g(X)], the outer Expected valueis with respect to the probability distribution [E(Y|X)] =E[g(X)]= g(x)fX(x)dx= ( y fY|X(y|x)dy)fX(x)dx26 / 31 Expected ValueVarianceCovarianceProof of the double expectation formulaE(Y) =E[E(Y|X)]E[E(Y|X)] = ( y fY|X(y|x)dy)fX(x)dx= yfX,Y(x,y)fX(x)dy fX(x)dx= y fX,Y(x,y)dy dx=E(Y)27 / 31 Expected ValueVarianceCovarianceDefinition of CovarianceLetXandYbe jointly distributed random variables withE(X) = xandE(Y) = y.

6 ThecovariancebetweenXandYisCov(X,Y) =E[(X X)(Y Y)]If values ofXthat are above average tend to go with valuesofYthat are above average (and below averageXtends togo with below averageY), the Covariance will be above average values ofXtend to go with values ofYthat arebelowaverage, the Covariance will be means they vary could think ofV ar(X) =E[(X X)2] asCov(X,X).28 / 31 Expected ValueVarianceCovarianceProperties of CovarianceCov(X,Y) =E(XY) E(X)E(Y)IfXandYare independent,Cov(X,Y) = 0 IfCov(X,Y) = 0, it doesnotfollow (a+X,b+Y)=Cov(X,Y)Cov(aX,bY)=abCov(X,Y)C ov(X,Y+Z)=Cov(X,Y) +Cov(X,Z)V ar(aX+bY) =a2V ar(X) +b2V ar(Y) + 2abCov(X,Y)IfX1,..,Xnare ar( ni=1Xi) = ni=1V ar(Xi)29 / 31 Expected ValueVarianceCovarianceCorrelationCorr(X ,Y) = =Cov(X,Y) V ar(X)V ar(Y) 1 1 Scale free:Corr(aX,bY) =Corr(X,Y)30 / 31 Expected ValueVarianceCovarianceCopyright InformationThis slide show was prepared by Jerry Brunner, Department ofStatistical Sciences, University of Toronto. It is licensed under aCreative Commons Attribution - ShareAlike UnportedLicense.

7 Use any part of it as you like and share the resultfreely. The LATEX source code is available from the coursewebsite: brunner/oldclass/256f1931 / 31