Continuous Random Variables Expected Values and Moments

1 Continuous Random VariablesExpected Values and MomentsStatistics 110 Summer 2006 Copyrightc 2006 by Mark E. IrwinContinuous Random VariablesWhen defining a distribution for a Continuous RV, the PMF approach won tquite work since summations only work for a finite or a countably infinitenumber of items. Instead they are based on the followingDefinition:LetXbe a Continuous RV. TheProbability Density Function(PDF) is a functionf(x)on the range ofXthat satisfies the (x) f(x) 0 fis piecewise Continuous f(x)dx= 1 Continuous Random Variables1 For anya < b, the probability thatP[a < X < b]is the area under thedensity curve (x)abP[a < X < b] = baf(x)dxContinuous Random Variables2 Note thatf(a)isNOTthe probability of observingX=aasP[X=a] = aaf(x)dx= 0 Thus the probability that a Continuous RV takes on any particular value is0.

2 (While this might seem counterintuitive, things do work properly.) Aconsequence of this is thatP[a < X < b] =P[a X < b] =P[a < X b] =P[a X b]for Continuous RVs. Note that this won t hold for discrete Random Variables3 Note that for small , iffis Continuous atxP[x 2 X x+ 2]= x+ 2x 2f(u)du f(x) Xf(x)x 2x+ 2xSo the probability of seeing an outcome in a small interval aroundxisproportional tof(x). So the PDF is giving information of how likely anobservation Random Variables4As with the PMF and the CDF for discrete RVs, there is a relationshipbetween the PDF,f(x), and the CDF,F(x), for Continuous RVsF(x) =P[X x] = x f(u)duf(x) =F (x)assuming thatfis Continuous on this relationship, the probability for any reasonable event describinga RV can determined with the CDF as the probability of any interval satisfiesP[a < X b] =F(b) F(a)Note that this is slightly different than the formula given on page 47.

3 Theabove holds for any RV (discrete, Continuous , mixed). The form given onpage 47P[a X b] =F(b) F(a)only holds for Continuous Random Variables5 Example: Uniform RV on [0,1] (DenotedX U(0,1))What most people think of when we say pick a numberbetween 0 and 1. Any real number in the interval ispossible and equally likely, implying that any interval oflengthhmust have the same probability (which needsto beh). The PDF forXthen must bef(x) ={1 0 x 10x <0orx >1 of U(0,1)xf(x) Continuous Random Variables6 The CDF for aU(0,1)isF(x) = 0x <0x0 x 11x >1 of U(0,1)xF(x) Continuous Random Variables7 One way to think of the CDF is that you give a value of the RV and it givesa probability associated with it ( [X x]).}

4 It can also be useful to gothe other way. Give a probability and figure out which value of the RV isassociated with assume thatFis Continuous and strictly increasing in some intervalI( 0to the left ofIandF= 1to the right ofI) (noteImightbe unbounded). Under these assumptions the inverse functionF 1is welldefined (x=F 1(y)ifF(x) =y).Definition:ThepthQuantileof the distributionFis defined to be thevaluexpsuch thatF(xp) =porP[X xp] =pUnder the above assumptionsxp=F 1(p). Continuous Random Variables8 4 (x)xppSpecial cases of interest of theMedian(p=12) and the lower and upperQuartiles(p=14and=34) Continuous Random Variables9 Note: Defining quantiles for discrete distributions is a bit tougher since theCDF doesn t take all Values between 0 and 1 (due to the jumps) for number of heads in 3 flipsx (number of heads)P[X <= x]The definition above can be extended to solving the simultaneous equationsP[X xp] pandP[X < xp] pContinuous Random Variables10 This can be though of as the place where the CDF jumps from belowptoabovep for number of heads in 3 flipsx (number of heads)P[X <= x] function for number of heads in 3 flipspxpContinuous Random Variables11 Expected Values and MomentsDefinition.

5 TheExpected Valueof a Continuous RVX(with PDFf(x))isE[X] = xf(x)dxassuming that |x|f(x)dx < .The Expected value of a distribution is often referred to as the mean of with the discrete case, the absolute integrability is a technical point,which if ignored, can lead to an example of a Continuous RV with infinite mean, see the Cauchydistribution (Example G, page 114) Expected Values and Moments12As with the discrete case,E[X]can be thought as a measure of center ofthe Random example, whenX U(0,1)E[X] = 10xdx= of U(0,1)xf(x) Expected Values and Moments13 Not surprisingly, expectations of functions of Continuous RVs satisfy theexpected relationshipE[g(X)] = g(x)f(x)dxFor example, ifX U(0,1)

6 ,E[X2] = 10x2dx=13 This is often easier than figuring out the PDF ofY=g(X)and applyingthe definition as there is often some work to figure out the PDF ofY.(Which we will do later, it does have its uses)As with discrete RVs,g(E[X])6=E[g(X)]in most cases. However, with alinear transformationY=a+bXE[a+bX] =a+bE[X] Expected Values and Moments14 Spread of a RV 2 1012XP[X=x] 2 1012XP[X=x] 0 1p(x)131313x-2 -1 0 1 2p(x)1929392919 Expected Values and Moments15 2 (x) 2 (x)f(x) ={ 1 x 10 Otherwisef(x) = +x4 2 x x40 x 20 OtherwiseAll these distributions haveE[X] = 0but the right hand side in each casehas a bigger spread.}

7 A common measure of spread is the Standard DeviationExpected Values and Moments16 Definition:Let =E[X], then theVarianceof the Random variableXisVar(X) =E[(X )2]provided the expectation DeviationofXisSD(X) = Var(X)For a discrete RV,Var(X) = i(xi )2p(xi)For a Continuous RVVar(X) = (x )2f(x)dxExpected Values and Moments17 The variance measures the Expected squared difference of an observationfrom the mean. While the interpretation of the standard deviation isn tquite easy, it can be thought of a measure of the typical spread of a can be shown that, assuming that the variance exists,Var(X) =E[X2] (E[X])2 This form is often useful for calculation : The variance is often denoted by 2and the standard deviationby.

8 Expected Values and Moments18 For the examples 2 1012XP[X=x] 1 0 1p(x)131313 Var(X) = ( 1 0)213+ (0 0)213+ (1 0)213=23SD(X) = 23= Values and Moments19 2 1012XP[X=x] 2 1 0 1 2p(x)1929392919 Var(X) = ( 2 0)219+ ( 1 0)229+ (0 0)239+ (1 0)229+ (2 0)219=109SD(X) = 109= Values and Moments20 2 (x)f(x) ={ 1 x 10 OtherwiseVar(X) = 1 1(x 0)212dx=13SD(X) = 13= Values and Moments21 2 (x)f(x) = +x4 2 x x40 x 20 OtherwiseVar(X) = 2 20(x 0)2( x4)dx=43SD(X) = 43= Values and Moments22 What is the effect of a linear transformation (Y=a+bX) on the varianceand standard deviation?}

9 Var(a+bX) =b2 Var(X) SD(a+bX) =|b|SD(X)These two results are to be Expected . For example, if two possibleXvaluesdiffer byd=|x1 x2|, the correspondingYvalues differ by|b|d, suggestingthat we want the standard deviation to scale by a factor of|b|. Since thevariance measures squared spread, it needs to scale by a factor factoranot having an effect also makes sense. Addingato a randomvariable shifts the location of its distribution, but doesn t changes thedistance between corresponding pairs of Values and Moments23