Chapter 3: Expectation and Variance

1 44 Chapter 3: Expectation and VarianceIn the previous Chapter we looked at probability, with threemajor themes:1. Conditional probability:P(A|B).2. First-step analysis for calculating eventual probabilities in a Calculating probabilities for continuous and discrete random this Chapter , we look at the same themes Expectation of a random variable is thelong-term average of the observing many thousands of independent random values from therandom variable of interest. Take the average of these random values. Theexpectation is the value of this average as the sample size tends to will repeat the three themes of the previous Chapter , but in a different Calculating expectations for continuous and discrete random Conditional Expectation : the Expectation of a random variableX,condi-tionalon the value taken by another random variableY.

2 If the value ofYaffects the value ofX( ), the conditionalexpectation ofXgiven the value ofYwill be different from the overallexpectation First-step analysis for calculating the expected amountof time needed toreach a particular state in a process ( the expected number of shotsbefore we win a game of tennis).We will also study similar themes for ,expected value, orexpectationof a random variableXis writ-ten asE(X) or X. If we observeNrandom values ofX, then the mean of theNvalues will be approximately equal toE(X) for largeN. The Expectation isdefined differently for continuous and discrete random :LetXbe acontinuousrandom variable with (x).

3 The ex-pected value ofXisE(X) =Z xfX(x) :LetXbe adiscreterandom variable with probability functionfX(x).The expected value ofXisE(X) =XxxfX(x) =XxxP(X=x). Expectation ofg(X)Letg(X) be a function ofX. We can imagine a long-term average ofg(X) justas we can imagine a long-term average ofX. This average is written asE(g(X)).Imagine observingXmany times (Ntimes) to give resultsx1, x2, .. , xN. Applythe functiongto each of these observations, to giveg(x1), .. , g(xN). The meanofg(x1), g(x2), .. , g(xN) approachesE(g(X)) as the number of observationsNtends to :LetXbe acontinuousrandom variable , and letgbe a function. Theexpected value ofg(X) isE g(X) =Z g(x)fX(x) :LetXbe adiscreterandom variable , and letgbe a function.

4 Theexpected value ofg(X) isE g(X) =Xxg(x)fX(x) =Xxg(x)P(X=x).46 Expectation ofXY: the definition ofE(XY)Suppose we have two random variables,XandY. These might be independent,in which case the value ofXhas no effect on the value ofY. Alternatively,XandYmight bedependent: when we observe a random value forX, itmight influence the random values ofYthat we are most likely to observe. Forexample,Xmight be the height of a randomly selected person, andYmightbe the weight. On the whole, larger values ofXwill be associated with largervalues understand whatE(XY) means, think of observing a large number ofpairs(x1, y1),(x2, y2), .. ,(xN, yN). IfXandYare dependent, the valueximightaffect the valueyi, and vice versa, so we have to keep the observations togetherin their pairings.

5 As the number of pairsNtends to infinity, the average1 NPNi=1xi yiapproaches the expectationE(XY).For example, ifXis height andYis weight,E(XY) is the average of (height weight). We are interested inE(XY) because it is used for calculating thecovarianceandcorrelation, which are measures of how closely relatedXandYare (see Section ).Properties of Expectationi) Letgandhbe functions, and letaandbbe constants. For any random variableX(discrete or continuous),Enag(X) +bh(X)o=aEng(X)o+bEnh(X) particular,E(aX+b) =aE(X) + ) LetXandYbe ANY random variables (discrete, continuous, independent, ornon-independent). ThenE(X+Y) =E(X) +E(Y).More generally, for ANY random variablesX1.

6 , Xn,E(X1+..+Xn) =E(X1) +..+E(Xn).47iii) LetXandYbeindependentrandom variables, andg, hbe functions. ThenE(XY) =E(X)E(Y)E g(X)h(Y) =E g(X) E h(Y) . (XY) =E(X)E(Y) is ONLY generally true IfXandYare independent, thenE(XY) =E(X)E(Y). However, theconverse is not generally true: it is possible forE(XY) =E(X)E(Y) eventhoughXandYare as an ExpectationLetAbe any event. We can writeP(A) as an Expectation , as theindicator function:IA=(1if eventAoccurs, arandom variable , andE(IA) =1Xr=0rP(IA=r)= 0 P(IA= 0) + 1 P(IA= 1)=P(IA= 1)=P(A).ThusP(A) =E(IA)for any , covariance, and correlationThe Variance of a random variableXis a measure of howspread outit the values ofXclustered tightly around their mean, or can we commonlyobserve values ofXa long way from the mean value?

7 Thevariancemeasureshow far the values ofXare from their mean, on :LetXbe any random variable . ThevarianceofXisVar(X) =E (X X)2 =E(X2) E(X) Variance is themean squared deviationof a random variable from its Variance , we can observe values ofXa long way from the Variance , the values ofXtend to be clustered tightly around themean :LetXbe a continuous random variable with (x) =(2x 2for 1< x <2, (X) and Var(X).E(X) =Z x fX(x)dx=Z21x 2x 2dx=Z212x 1dx=h2 log(x)i21= 2 log(2) 2 log(1)= 2 log(2).49 For Var(X), we useVar(X) =E(X2) {E(X)} (X2) =Z x2fX(x)dx=Z21x2 2x 2dx=Z212dx=h2xi21= 2 2 2 1= (X) =E(X2) {E(X)}2= 2 {2 log(2)}2= is a measure of the association or dependence between two randomvariablesXandY.)

8 Covariance can be either positive or negative. (Varianceisalways positive.)Definition:LetXandYbe any random variables. ThecovariancebetweenXandYis given bycov(X, Y) =En(X X)(Y Y)o=E(XY) E(X)E(Y),where X=E(X), Y=E(Y).1. cov(X, Y) will bepositiveif large values ofXtend to occur with large valuesofY, and small values ofXtend to occur with small values ofY. For example,ifXis height andYis weight of a randomly selected person, we would expectcov(X, Y) to be cov(X, Y) will benegativeif large values ofXtend to occur with small valuesofY, and small values ofXtend to occur with large values ofY. For example,ifXis age of a randomly selected person, andYis heart rate, we would expectXandYto be negatively correlated (older people have slower heartrates).

9 3. IfXandYare independent, then there is no pattern between large values ofXand large values ofY, so cov(X, Y) = 0. However, cov(X, Y) = 0 does NOTimply thatXandYare independent, unlessXandYare Normally of Variancei) Letgbe a function, and letaandbbe constants. For any random variableX(discrete or continuous),Varnag(X) +bo=a2 Varng(X) particular,Var(aX+b) =a2 Var(X).ii) LetXandYbeindependentrandom variables. ThenVar(X+Y) =Var(X) +Var(Y).iii) IfXandYareNOT independent, thenVar(X+Y) =Var(X) +Var(Y) + 2cov(X, Y).Correlation (non-examinable)The correlation coefficient ofXandYis a measure of the linear associationbetweenXandY. It is given by the covariance, scaled by the overall variabilityinXandY.

10 As a result, the correlation coefficient is always between 1 and+1, so it is easily compared for different :ThecorrelationbetweenXandY, also called thecorrelation coefficient,is given bycorr(X, Y) =cov(X, Y)pVar(X)Var(Y).51 The correlation measures linearassociation betweenXandY. It takes valuesonly between 1 and +1, and has the same sign as the correlation is 1 if and only if there is a perfect linear relationship betweenXandY, corr(X, Y) = 1 Y=aX+bfor some correlation is 0 ifXandYare independent, but a correlation of 0 doesnotimplythatXandYare Expectation and Conditional VarianceThroughout this section, we will assume for simplicity thatXandYare dis-crete random variables.