5. Convergence of sequences of random variables

1 5. Convergence of sequences ofrandom variablesThroughout this chapter we assume that{X1,X2,..}is a sequence of andXis , and all of them are defined on the same probability space ( ,F,P).Stochastic Convergence formalizes the idea that a sequence of sometimes isexpected to settle into a pattern may for instance be that: there is a Convergence ofXn( ) in the classical sense to a fixed valueX( ), for eachand every event ; the probability that the distance betweenXnfrom a particular anyprescribed positive value decreases and converges to zero; the series formed by calculating the expected value of the (absolute or quadratic)distance betweenXnandXconverges to zero.

2 The distribution ofXnmay grow increasingly similar to the distribution of aparticular as in analysis, we can distinguish among several types of Convergence (Rohatgi,1976, p. 240). Thus, in this chapter we investigate modes of Convergence of sequences : almost sure Convergence ( ); Convergence in probability (P ); Convergence in quadratic mean or inL2( ); Convergence inL1or in mean (L1 );1 See Convergence in distribution (d ).Two laws of large numbers and central limit theorems are also is important for the reader to be familiarized with all these modes of Convergence ,the way they can be related and with the applications of such results and understandtheir considerable significance in probability, statistics and stochastic Modes of convergenceThe first four modes of Convergence ( , where = ,P, ,L1) pertain to thesequence of and toXas functions of , while the fifth (d ) is related to the convergenceof (Karr, 1993, p.)

3 135). Convergence of as functions on Motivation Almost sure Convergence (Karr, 1993, p. 135)Almost sure Convergence or Convergence with probability one is the probabilisticversion of pointwise Convergence known from elementary real analysis. Definition Almost sure Convergence (Karr, 1993, p. 135; Rohatgi, 1976, )The sequence of {X1,X2,..}is said to converge almost surely to a ({w: limn + Xn( ) =X( )})= 1.( )In this case we X(orXn Xwith probability 1). Exercise Almost sure convergenceLet{X1,X2,..}be a sequence of such thatXn Bernoulli(1n), n 0, by derivingP({Xn= 0,for everym n n0}) and observingthat this probability does not converge to 1 asn0 + for all values ofm(Rohatgi,1976, p.

4 252, Example 9). 217 Motivation Convergenceinprobability(Karr,1993, ; ) Convergence in probability essentially means that the probability that|Xn X|exceedsany prescribed, strictly positive value converges to basic idea behind this type of Convergence is that the probability of an unusual outcome becomes smaller and smaller as the sequence progresses. Definition Convergence in probability(Karr, 1993, p. 136; Rohatgi, 1976, )The sequence of {X1,X2,..}is said to converge in probability to a denotedbyXnP X iflimn + P({|Xn X|> }) = 0,( )for every >0. Remarks Convergenceinprobability(Rohatgi,1976, ; ) The definition of Convergence in probability says nothing about the convergenceof the sense in which it is understood in real analysis.

5 Thus,XnP Xdoes not imply that, given >0, we can find anNsuch that|Xn X|< ,forn speaks only of the Convergence of the sequence of probabilitiesP(|Xn X|> ) to zero. Formally, Definition means that , >0, N :P({|Xn X|> })< , n N .( ) The concept of Convergence in probability is used very often in statistics. Forexample, an estimator is called consistent if it converges in probability to theparameter being estimated. Convergence in probability is also the type of Convergence established by the weaklaw of large numbers. 218 Exercise Convergence in probabilityLet{X1,X2.}

6 }be a sequence of such thatXn Bernoulli(1n), n IN.(a) Prove thatXnP 0, by obtainingP({|Xn|> }), for 0< <1 and 1 (Rohatgi,1976, pp. 243 244, Example 5).(b) Verify thatE(Xkn) E(Xk), wherek INandXd= 0. Exercise Convergence in probability does not imply Convergence {X1,X2,..}be a sequence of such thatXnd=n Bernoulli(1n), n IN, ({Xn=x}) = 1 1n, x= 01n, x=n0,otherwise.( )Prove thatXnP 0, howeverE(Xkn)6 E(Xk), wherek INand the degenerateat 0 (Rohatgi, 1976, p. 247, Remark 3). Motivation Convergence in quadratic mean and inL1We have just seen that Convergence in probability does not imply the Convergence ofmoments, namely of orders 2 or 1.

7 Definition Convergence in quadratic mean or inL2(Karr, 1993, p. 136)LetX,X1,X2,..belong toL2. Then the sequence of {X1,X2,..}is said to convergetoXin quadratic mean (or inL2) denoted X(orXnL2 X) iflimn + E[(Xn X)2]= 0.( ) Exercise Convergence in quadratic meanLet{X1,X2,..}be a sequence of such thatXn Bernoulli(1n).Prove X, where the degenerate at 0 (Rohatgi, 1976, p. 247,Example 6). 219 Exercise Convergence in quadratic mean (bis)Let{X1,X2,..}be a sequence of withP({Xn= 1n})= X, where the degenerate at 0 (Rohatgi, 1976, p.)

8 252,Example 11). Exercise Convergence in quadratic mean implies Convergence of (Karr, 1993, p. 158, Exercise (a))Prove X E(X2n) E(X2) (Rohatgi, 1976, p. 248, proof of Theorem 8). Exercise Convergence in quadratic mean of partial sums(Karr, 1993, , Exercise )LetX1,X2,..be pairwise uncorrelated with mean zero and partial sumsSn= ni= that if there is a constantcsuch thatV(Xi) c, for everyi, thenSnn 0 for all >12. Definition Convergence in mean or inL1(Karr, 1993, p. 136)LetX,X1,X2,..belong toL1. Then the sequence of {X1,X2.}

9 }is said to convergetoXin mean (or inL1) denoted byXnL1 X iflimn + E(|Xn X|) = 0.( ) Exercise Convergence in mean implies Convergence of 1st. moments(Karr, 1993, p. 158, Exercise (b))Prove thatXnL1 X E(Xn) E(X). Convergence in distributionMotivation Convergence in distribution( ) Convergence in distribution is very frequently used in practice, most often it arises fromthe application of the central limit theorem. Definition Convergence in distribution(Karr, 1993, p. 136; Rohatgi, 1976,pp. 240 1)The sequence of {X1,X2,..}converges toXin distribution denoted byXnd X iflimn + FXn(x) =FX(x),( )for allxat whichFXis continuous.

10 Remarks Convergence in distribution( ; Karr, 1993, p. 136;Rohatgi, 1976, p. 242) With this mode of Convergence , we increasingly expect to see the next in asequence of becoming better and better modeled by a given , as seen inExercise It must be noted that it is quite possible for a given sequence of to converge toa function that is not a , as shown in Exercise The requirement that only the continuity points ofFXshould be considered isessential, as we shall see in exercises and The Convergence in distribution does not imply the Convergence of correspondingp.