Transcription of Convergence in Distribution Central Limit Theorem
1 Convergence in DistributionCentral Limit TheoremStatistics 110 Summer 2006 Copyrightc 2006 by Mark E. IrwinConvergence in Bin(n, p)and let =np, Thenlimn P[X=x] = limn (nx)px(1 p)n x=e xx!So whenngets large, we can approximate binomial probabilities withPoisson (nx)px(1 p)n x= limn (nx)( n)x(1 n)n x=n!x!(n x)! x(1nx)(1 n) x(1 n)nConvergence in Distribution1=n!x!(n x)! x(1nx)(1 n) x(1 n)n= xx!limn n!(n x)!1(n )x 1(1 n)n e =e xx!2 Note that approximation works better whennis large andpis small ascan been seen in the following plot.
2 Ifpis relatively large, a differentapproximation should be used. This is coming later.(Note in the plot, bars correspond to the true binomial probabilities and thered circles correspond to the Poisson approximation.) Convergence in Distribution201234lambda = 1 n = 10 p = (x) = 1 n = 50 p = (x) = 1 n = 200 p = (x) 1 2 3 4 5 6 7 8 9lambda = 5 n = 10 p = (x) = 5 n = 50 p = (x) = 5 n = 200 p = (x) in Distribution3 Example: LetY1, Y2.
3 Be iidExp(1). ThenXn=Y1+Y2+..+Yn Gamma(n,1)which hasE[Xn] =n; Var(Xn) =n;SD(Xn) = nThusZn=Xn n nhas mean = 0 and variance = compare its Distribution toZ N(0,1). IsP[ 1 Zn 2] P[ 1 Z 2]?LetZn=Xn n n;Xn=n+ nZnfZn(z) =fXn(n+ nz) nConvergence in Distribution4P[a Zn b] = bafZn(z)dz= ba nfXn(n+ nz)dz= ba n(n+ nz)n 1(n 1)!e (n+ nz)dzTo go further we need Stirling s Formula:n! nne n 2 n. SofXn(n+ nz) n=e n z n(n+z n)n 1 n(n 1)! e n z n(n+z n)n 1 n(n 1)n 1e n+1 2 n 1 2 e z n(1 +z n)n gn(z) Convergence in Distribution5log(gn(z)) = z n+nlog(1 +z n)= z n+n[z n 12z2n+13z3n3/2.]
4 ] 12z2+O(1 n)sofXn(n+z n) n 1 2 e z2/2 ThusP[a Zn b] ba1 2 e z2/2dz=P[a Z b]So asnincreases, the Distribution ofZngets closer and closer to aN(0,1). Convergence in Distribution6 Another way of thinking of this, is that the Distribution ofXn=n+Zn napproaches that of aN(n, n). = 2xf(x) 20246810 = 5xf(x) = 10xf(x) = 20xf(x) = 50xf(x)708090100 110 120 = 100xf(x) Convergence in , X2, ..be a sequence of RVs with cumulativedistribution functionsF1, F2, ..and letXbe a RV with distributionF.
5 We sayXnConverges in DistributiontoXiflimn Fn(x) =F(x)at every point at whichFis XAn equivalent statement to this is that for allaandbwhereFis continuousP[a Xn b] P[a X b]Note that ifXnandXare discrete distributions, this condition reduces toP[Xn=xi] P[X=xi]for all support in Distribution8 Note that an equivalent definition of Convergence in Distribution is thatXnD XifE[g(Xn)] E[g(X)]for all bounded, continuous functionsg( ).This statement of Convergence in Distribution is needed to help prove thefollowing theoremTheorem.
6 [Continuity Theorem ]LetXnbe a sequence of randomvariables with cumulative Distribution functionsFn(x)and correspondingmoment generating functionsMn(t). LetXbe a random variable withcumulative Distribution functionF(x)and moment generating functionM(t). IfMn(t) M(t)for alltin an open interval containing zero, thenFn(x) F(x)at all continuity points ofF. That isXnD the previous two examples (Binomial/Poisson and Gamma/Normal)could be proved this in Distribution9 For the Gamma/Normal exampleMZn(t) =MXn(t n)e t n=(11 t n)ne t nSimilarly to the earlier proof, its easier to work withlogMZn(t)logMZn(t) = t n nlog(1 t n)= t n n[ t n 12t2n 13t3n3/2.]
7 ]=12t2+O(1 n)ThusMZn(t) et2/2which is the MGF for a standard in Distribution10 Central Limit TheoremTheorem. [ Central Limit Theorem (CLT)]LetX1, X2, X3, ..be asequence of independent RVs having mean and variance 2and acommon Distribution functionF(x)and moment generating functionM(t)defined in a neighbourhood of zero. LetSn=n i=1 XnThenlimn P[Sn n n x]= (x)That isSn n nD N(0,1) Central Limit a loss of generality, we can assume that = 0. So letZn=Sn n. SinceSnis the sum ofniid RVs,MSn(t) = (M(t))n;MZn(t) =(M(t n))nTaking a Taylor series expansion ofM(t)around 0 givesM(t) =M(0) +M (0)t+12M (0)t2+ t= 1 +12 2t2+O(t3)sinceM(0) = 1, M (0) = = 0, M (0) = 2.
8 SoM(t n)= 1 +12 2(t n)2+O((t n)3)= 1 +t22n+O(1n3/2) Central Limit Theorem12 This givesMZn(t) =(1 +t22n+O(1n3/2))n et2/22 Note that the requirement of a MGF is not needed for the Theorem to fact, all that is needed is thatVar(Xi) = 2< . A standard proof ofthis more general Theorem uses the characteristic function (which is definedfor any Distribution ) (t) = eitxf(x)dx=M(it)instead of the moment generating functionM(t), wherei= the CLT holds for distributions such as the log normal, even thoughit doesn t have a Limit Theorem13 Also, the CLT is often presented in the following equivalent formZn= Xn / n= n Xn D N(0,1)
9 To see this is the same, just multiply the numerator and denominator bynin the first form to get the statement common way that this is used is thatSnapprox. N(n , n 2)or Xnapprox. N( , 2n) Central Limit Theorem14 Example: Insurance claimsSuppose that an insurance company has 10,000 policy holders. The expectedyearly claim per policyholder is $240 with a standard deviation of $ is the approximate probability that the total yearly claimsS10,000>$ MillionE[S10,000] = 10,000 240 = 2,400,000SD(S10,000) = 10,000 800 = 80,000P[S10,000>2,600,000]=P[S10,000 2,400,00080,000>2,600,000 2,400,00080,000] P[Z > ] = that this probability statement does not use anything about thedistribution of the original policy claims except their mean and standarddeviation.
10 Its probable that their Distribution is highly skewed right (since x<< x), but the calculations ignore this Limit Theorem15 One consequence of the CLT is the normal approximation to the Bin(n, p)and pn=Xnn, then (sinceXncan be thought of the sumofnBernoulli s)Xn np np(1 p)D N(0,1); pn p p(1 p)/nD N(0,1)Another way of think of this is thatXnapprox. N(np, np(1 p)); pnapprox. N(p,p(1 p)n)This approximation works better whenpis closer to12than whenpis near0 or rule of thumb is that is ok to use the normal approximation whennp 5andn(1 p) 5(expect at least 5 successes and 5 failures).
