Lecture 6 Moment-generating functions

1 Lecture6: Moment-generating functions1of11 Course:Mathematical StatisticsTerm:Fall2017 Instructor:Gordan itkovi cLecture6 Moment-generating and first propertiesWe use many different functions to describe probability distribution (pdfs,pmfs, cdfs, quantile functions , survival functions , hazard functions , etc.) Moment-generating functions are just another way of describing distribu-tions, but they do require getting used as they lack the intuitive appeal ofpdfs or function (mgf)of the (dis-tribution of the) random variableYis the functionmYof a real param-etertdefined bymY(t) =E[etY],for allt Rfor which the expectationE[etY]is well is hard to give a direct intuition behind this definition, or to explain atwhy it is useful, at this point.

2 It is related to the notions of Fourier transformand generating functions . It will be only through examples in this and laterlectures that a deeper understanding will first order of business is to compute the mgf for some of the more im-portant (named) random variables. In the case of a continuous distribution,the main tool is the fundamental theorem which we use with the functiong(y) =exp(ty)- we think oftas fixed, so thatmY(t) =E[exp(tY)] =E[g(Y)] = g(y)fY(y)dy= etyfY(y) U(0, 1), so thatfY(y) =1{0 y 1}.Last Updated:September25,2019 Lecture6: Moment-generating functions2of11 ThenmY(t) = etyfY(y)dy= 10etydy=1t(et 1). us compute the mgf of the exponen-tial distributionY E( )with parameter >0:mY(t) = 0ety1 e y/ dy=1 0e y(1 t)dy=1 11 t=11 N(0, 1).

3 As above,mY(t) = ety1 2 e integral looks hard to evaluate, but there is a simple trick. Wecollect the exponential terms and complete the square:etye 12y2=e 12(y t) we plug this into the expression above and pull oute12t2which isconstant, as far as the variable of integration is concerned, we getmY(t) =e12t2 1 2 e 12(y t) does not look like a big improvement at first, but it is. Theexpression inside the integral is the pdf of a normal distributionwith meantand variance 1. Therefore, it must integrate to 1, asdoes any pdf. It follows thatmY(t) = you can see from the first part of this example, the moment generatingfunction does not have to be defined for allt.

4 Indeed, the mfg of the expo- nential function is defined only fort<1 . We will not worry too much forabout this, and simply treat mgfs as expressions int, but this fact is good tokeep in mind when one goes deeper into the fundamental formula for continuous distributions becomes a sum inthe discrete case. WhenYis discrete with supportSYand pmfpY, the mgfcan be computed as follows, where, as above,g(y) =exp(ty):mY(t) =E[etY] =E[g(Y)] = y SYexp(ty)pY(y).Last Updated:September25,2019 Lecture6: Moment-generating functions3of11 Here are some Bernoulli B(p), we havemY(t) =et 0pY(0) +et 1pY(1) =q+pet,whereq=1 p. Geometric g(p), thenP[Y=y] =qypandsomY(t) = y=0etypqy=p y=0(qet)y=p1 qet,where the last equality uses the familiar expression for the sumof a geometric series.

5 We note that this only works forqet<1,so that, like the exponential distribution, the geometric distri-bution comes with a mgf defined only for some values oft. Poisson the Poisson distributionP( )with parameter >0, so thatSY={0, 1, 2, 3, ..}andpY(y) =e yy!. ThenmY(t) =E[exp(tY)] =E[g(Y)] = y SYg(y)pY(y)= y=0exp(ty)exp( ) yy!=e y=0(et )yy!The last sum on the right is nothing else by the Taylor formulafor the exponential function atx=et . Therefore,mY(t) =e (et 1).Here is how to compute the moment generating function of a linear trans-formation of a random variable. The formula follows from the simple factthatE[exp(t(aY+b))] =etbE[e(at)Y] that the random variable Y has the mgf mY(t).

6 Then mgf of the random variable W=aY+b, where a and b are constants,is given bymW(t) =etbmY(at).Last Updated:September25,2019 Lecture6: Moment-generating U(l,r). We can representWasW=aY+bwhereY U(0, 1),a= (r l)andb=l. We computedthe mgf ofYin -mY(t) =1t(et 1). Therefore,by (t) =etl1(r l)t(e(r l)t 1) =etr etat(r l). N( , ), thenWhas the same distri-bution as + Z, whereZ N(0, 1). Using the expression the mgf of aunitnormal distributionZ N(0, 1),we havemW(t) =e te12 2t2=e t+12 of independent random variablesOne of the most important properties of the Moment-generating functions isthat they turn sums of independent random variables into Y1,Y2.

7 ,Ynbe independent random variables withmgfs mY1(t), mY2(t), .. , mYn(t). Then the mgf of their sum Y=Y1+Y2+ +Ynis given bymY(t) =mY1(t) mY2(t) mYn(t).This proposition is true for all random variables, but here is a sketch ofthe argument in the continuous case. It is a consequence of the factoriza-tion theorem (Theorem??) and the fundamental formula (Theorem??). Forsimplicity, let us assume thatn=2:mY1+Y2(t) =E[et(Y1+Y2)] =E[g(Y1,Y2)],whereg(y1,y2) =et(y1+y2). The factorization criterion says thatfY1,Y2(y1,y2) =Last Updated:September25,2019 Lecture6: Moment-generating functions5of11fY1(y1)fY2(y2), and, somY1+Y2(y) = et(y1+y2)fY1,Y2(y1,y2)dy2dy1= ety1ety2fY1(y1)fY2(y2)dy2dy1= ety1fY1(y1)( ety2fY2(y2)dy2)dy1= ety1fY1(y1)mY2(t)dy2=mY2(t) ety2fY2(y2)dy2=mY2(t)mY1(t).

8 Binomial b(n,p). We know thatYcounts the number of successes innindependent Bernoulli trials, sowe can represent (in distribution) asY=Y1+ +Yn, where eachYiis aB(p)-random variable. We know from the mgfmYi(t)of eachYiisq+pet. ThereforemY(t) =mY1(t) mY2(t) mYn(t) = (q+pet) could have obtained the same formula without the factorizationcriterion, but the calculation is trickier:mY(t) =n y=0etypY(y) =n y=0ety(ny)pyqn y=n y=0(ny)(pet)yqn y= (pet+q)n,where the last inequality follows from thebinomial formula(a+b)n=n y=0(ny)aybn Moment-generating ?The terminology moment generating function comes from the followingnice that the Moment-generating function mY(t)ofa random variable Y admits an expansion into a power series.

9 Then theLast Updated:September25,2019 Lecture6: Moment-generating functions6of11coefficients are related to the moments of Y in the following way:mY(t) = k=0 kk!tk,( )where k=E[Yk]is the k-th moment of fully rigorous argument of this proposition is beyond the scope of thesenotes, but we can see why it works is we do the following formal computationbased on the Taylor expansion of the exponential functionex= k=0xkk!.We plug inx=tYand then take the expectation to getmY(t) =E[etY] =E[ k=0(tY)kk!] = k=0tk1k!E[Yk] = k=0 kk! ( ) suggests the following approach to the computation of mo-ments of a random variable:1. Compute the mgfmY(t).2. Expand it in a power series int, , writemY(t) = k= Set k=k!

10 Of the exponential know from the mgfmY(t)of the exponentialE( )-distribution is11 t. It is not hard to expand this into a power series because11 tis nothing by the sum of a geometric series11 t= k=0 follows immediately that k=k! Updated:September25,2019 Lecture6: Moment-generating of the uniform same example as abovetells us that the mgf of the uniform distributionU(l,r)ismY(t) =etr etlt(r l).We expand this into a series, buy expanding the numerator, first:etr etl= k=01k!(tr)k k=01k!(tl)k= k=0rk lkk! we divide by the denominatort(r l)to getmY(t) = k=0rk lkk!(r l)tk 1=1+r2 l22!(r l)t+r3 l33!(r l)t2+..It follows that k=rk+1 lk+1(k+1)(r l).