CONDITIONAL EXPECTATION AND MARTINGALES

1 CONDITIONAL EXPECTATION AND MARTINGALES1. DI S CR E T E-TI MEMART I N GA L E of a {Fn}n 0be an increasing sequence of algebras in aprobability space ( ,F,P). Such a sequence will be called afiltration. LetX0,X1, .. be anadaptedsequence ofintegrablereal-valued random variables, that is, a sequence with the prop-erty that for eachnthe random variableXnis measurable relative toFnand such thatE|Xn|< . The sequenceX0,X1, .. is said to be amartingalerelative to the filtration {Fn}n 0if it isadapted and if for everyn,(1)E(Xn+1|Fn)= , it is said to be asupermartingale(respectively,submarting ale) if for everyn,(2)E(Xn+1|Fn) ( ) that any martingale is automatically both a submartingale and a and Martingale Difference most basic examples of martin-gales are sums of independent, mean zero random variables.

2 LetY0,Y1, .. be such a sequence;then the sequence of partial sums(3)Xn=n j=1 Yjis a martingale relative to the natural filtration generated by the variablesYn. This is easilyverified, using the linearity and stability properties and the independence law for conditionalexpectation:E(Xn+1|Fn)=E(Xn+Y n+1|Fn)=E(Xn|Fn)+E(Yn+1|Fn)=Xn+E Yn+1= importance of MARTINGALES in modern probability theory stems at least in part from thefact that many of the essential properties of sums of independent, identically distributed ran-dom variables are inherited (with minor modification) by MARTINGALES .

3 As you will learn, thereare versions of the SLLN, the central Limit theorem , the Wald indentities, and the Chebyshev, markov , and Kolmogorov inequalities for MARTINGALES . To get some appreciation of why thismight be so, consider the decomposition of a martingale {Xn} as a partial sum process:(4)Xn=X0+n j=1 jwhere j=Xj Xj difference sequence { n}has the following properties: (a) therandom variable nis a function ofFn; and (b) for every n 0,(5)E( n+1|Fn)= is a trivial consequence of the definition of a martingale. Corollary {Xn}be a martingale relative to{Yn}, with martingale difference sequence{ n}.

4 Then for every n 0,(6)E Xn=E , if E X2n< for some n 1then for j n the random variables jare square-integrableand uncorrelated, and so(7)E X2n=E X20+n j=1E first property follows easaily from Proposition 1 and the EXPECTATION Law for con-ditional EXPECTATION , as these together imply thatE n=0 for eachn. Summing and using thelinearity of ordinary EXPECTATION , one obtains (6).The second property is only slightly more difficult. For ease of exposition let s assume thatX0=0. (The general case can then be deduced by re-indexing the random variables.) First,observe that for eachk nthe random variableXkis square-integrable, by the Jensen inequal-ity for CONDITIONAL EXPECTATION , sinceXk=E(Xn|Fk).

5 Hence, each of the terms jhas finitevariance, because it is the difference of two random variables with finite second moments, andso all of the products i jhave finite first moments, by the Cauchy-Schwartz inequality. Next,ifj k nthen jis measurable relative toFj; hence, by Properties (1), (4), (6), and (7) ofconditional EXPECTATION , ifj k nthenE j k+1=E E( j k+1|Y1,Y2, .. ,Yk)=E jE k+1|Y1,Y2, .. ,Yk)=E( j 0)= variance ofXnmay now be calculated in exactly the same manner as for sums of indepen-dent random variables with mean zero:E X2n=En j=1 j)2=En j=1n k=1 j k=n j=1n k=1E j k=n j=1E 2j+2 j<kE j k=n j=1E 2j+0.

6 Examples of L vy s any integrable random variable. Then the sequenceXndefined byXn=E(X|Fn) is a martingale, by the Tower Property of CONDITIONAL Walk ,Y1, .. be a sequence of independent, identically dis-tributed random variables such thatE Yn=0. Then the sequenceXn= nj=1 Yjis a martingale,as we have Moment again letY0,Y1, .. be a sequence of independent,identically distributed random variables such thatE Yn=0 andE Y2n= 2< . Then the se-quence(8)(n j=1Yj)2 2nis a martingale (again relative to the sequence 0,Y1,Y2, .. ). This is also easy to Ratio MARTINGALES : Bernoulli ,X1.

7 Be a sequence of indepen-dent, identically distributed Bernoulli-prandom variables, and letSn= nj=1Xj. Note thatSnhas the binomial-(n,p) distribution. Define(9)Zn=(qp)2Sn ,Z1, .. is a martingale relative to the usual sequence. Once again, this is easy to martingale {Zn}n 0is quite useful in certain random walk problems, as we have Ratio MARTINGALES in ,X1, .. be independent, identically dis-tributed random variables whose moment generating function ( )=E e Xiis finite for somevalue 6=0. Define(10)Zn=Zn( )=n j=1e Xj ( )=e Sn ( ) a martingale. (It is called alikelihood ratiomartingale because the random variableZnis the likelihood ratiod P /d P0based on the sampleX1,X2.)

8 ,Xnfor probability measuresP andP0in a certain exponential family.) ,Z1,Z2, .. be a Galton-Watson process whose off-spring distribution has mean >0. Denote by (s)=E sZ1the probability generating functionof the offspring distribution, and by the smallest nonnegative root of the equation ( )= .Proposition of the following is a nonnegative martingale:Mn:=Zn/ n;andWn:= the traditional Polya urn model, an urn is seeded withR0=r 1 red ballsandB0=b 1 black balls. At each stepn=1, 2, .. , a ball is drawn at random from the urn andthen returned along with a new ball of the same color.

9 LetRnandBnbe the numbers of redand black balls afternsteps, and let n=Rn/(Rn+Bn)be the fraction of red balls. Then nis amartingale relatve to the natural Functions and markov , surely enough, MARTINGALES also arise inconnection with markov chains; in fact, one of Doob s motivations in inventing them was toconnect the world of potential theory for markov processes with the classical theory of sumsof independent random ,Y0,Y1, .. be a markov chain on a denumerable statespaceYwith transition probability matrixP. A real-valued functionh:Y Ris calledhar-monicfor the transition probability matrixPif(11)Ph=h,equivalently, if for everyx Y,(12)h(x)= y Yp(x,y)h(y)=Exh(Y1).

10 HereExdenotes the EXPECTATION corresponding to the probability measurePxunder whichPx{Y0=x}=1. Notice the similarity between equation (12) and the equation for the stationarydistribution one is just thetransposeof the h is harmonic for the transition probability matrixPthen for every startingstate x Ythe sequence h(Yn)is a martingale under the probability measure his 800-page bookClassical Potential Theory and its Probabilistic Counterpartfor more on is once again nothing more than a routine calculation. The key is the markov prop-erty, which allows us to rewrite any CONDITIONAL EXPECTATION onY0,Fnas a CONDITIONAL expecta-tion onYn.