CONDITIONAL EXPECTATION AND MARTINGALES

1 CONDITIONAL EXPECTATION AND MARTINGALES1. DI S CR E T E-TI MEMART I N GA L E of a {Fn}n 0be an increasing sequence of algebras in aprobability space ( ,F,P). Such a sequence will be called afiltration. LetX0,X1, .. be anadaptedsequence ofintegrablereal-valued random variables, that is, a sequence with the prop-erty that for eachnthe random variableXnis measurable relative toFnand such thatE|Xn|< . The sequenceX0,X1, .. is said to be amartingalerelative to the filtration {Fn}n 0if it isadapted and if for everyn,(1)E(Xn+1|Fn)= , it is said to be asupermartingale(respectively,submarting ale) if for everyn,(2)E(Xn+1|Fn) ( ) that any martingale is automatically both a submartingale and a and Martingale Difference most basic examples of martin-gales are sums of independent, mean zero random variables.

2 LetY0,Y1, .. be such a sequence;then the sequence of partial sums(3)Xn=n j=1 Yjis a martingale relative to the natural filtration generated by the variablesYn. This is easilyverified, using the linearity and stability properties and the independence law for conditionalexpectation:E(Xn+1|Fn)=E(Xn+Y n+1|Fn)=E(Xn|Fn)+E(Yn+1|Fn)=Xn+E Yn+1= importance of MARTINGALES in modern probability theory stems at least in part from thefact that many of the essential properties of sums of independent, identically distributed ran-dom variables are inherited (with minor modification) by MARTINGALES : As you will learn, thereare versions of the SLLN, the Central Limit Theorem, the Wald indentities, and the Chebyshev,Markov, and Kolmogorov inequalities for MARTINGALES .

3 To get some appreciation of why thismight be so, consider the decomposition of a martingale {Xn} as a partial sum process:(4)Xn=X0+n j=1 jwhere j=Xj Xj difference sequence { n}has the following properties: (a) therandom variable nis a function ofFn; and (b) for every n 0,(5)E( n+1|Fn)= is a trivial consequence of the definition of a martingale. Corollary {Xn}be a martingale relative to{Yn}, with martingale difference sequence{ n}.Then for every n 0,(6)E Xn=E , if E X2n< for some n 1then for j n the random variables jare square-integrableand uncorrelated, and so(7)E X2n=E X20+n j=1E first property follows easaily from Proposition 1 and the EXPECTATION Law for con-ditional EXPECTATION , as these together imply thatE n=0 for eachn.

4 Summing and using thelinearity of ordinary EXPECTATION , one obtains (6).The second property is only slightly more difficult. For ease of exposition let s assume thatX0=0. (The general case can then be deduced by re-indexing the random variables.) First,observe that for eachk nthe random variableXkis square-integrable, by the Jensen inequal-ity for CONDITIONAL EXPECTATION , sinceXk=E(Xn|Fk). Hence, each of the terms jhas finitevariance, because it is the difference of two random variables with finite second moments, andso all of the products i jhave finite first moments, by the Cauchy-Schwartz inequality. Next,ifj k nthen jis measurable relative toFj; hence, by Properties (1), (4), (6), and (7) ofconditional EXPECTATION , ifj k nthenE j k+1=E E( j k+1|Y1,Y2.)

5 ,Yk)=E jE k+1|Y1,Y2, .. ,Yk)=E( j 0)= variance ofXnmay now be calculated in exactly the same manner as for sums of indepen-dent random variables with mean zero:E X2n=En j=1 j)2=En j=1n k=1 j k=n j=1n k=1E j k=n j=1E 2j+2 j<kE j k=n j=1E 2j+0. Examples of L vy s any integrable random variable . Then the sequenceXndefined byXn=E(X|Fn) is a martingale, by the Tower Property of CONDITIONAL Walk ,Y1, .. be a sequence of independent, identically dis-tributed random variables such thatE Yn=0. Then the sequenceXn= nj=1 Yjis a martingale,as we have Moment again letY0,Y1, .. be a sequence of independent,identically distributed random variables such thatE Yn=0 andE Y2n= 2< . Then the se-quence(8)(n j=1Yj)2 2nis a martingale (again relative to the sequence 0,Y1,Y2.

6 This is also easy to Ratio MARTINGALES : Bernoulli ,X1, .. be a sequence of indepen-dent, identically distributed Bernoulli-prandom variables, and letSn= nj=1Xj. Note thatSnhas the binomial-(n,p) distribution. Define(9)Zn=(qp)2Sn ,Z1, .. is a martingale relative to the usual sequence. Once again, this is easy to martingale {Zn}n 0is quite useful in certain random walk problems, as we have Ratio MARTINGALES in ,X1, .. be independent, identically dis-tributed random variables whose moment generating function ( )=E e Xiis finite for somevalue 6=0. Define(10)Zn=Zn( )=n j=1e Xj ( )=e Sn ( ) a martingale. (It is called alikelihood ratiomartingale because the random variableZnis the likelihood ratiod P /d P0based on the sampleX1,X2.)

7 ,Xnfor probability measuresP andP0in a certain exponential family.) ,Z1,Z2, .. be a Galton-Watson process whose off-spring distribution has mean >0. Denote by (s)=E sZ1the probability generating functionof the offspring distribution, and by the smallest nonnegative root of the equation ( )= .Proposition of the following is a nonnegative martingale:Mn:=Zn/ n;andWn:= the traditional Polya urn model, an urn is seeded withR0=r 1 red ballsandB0=b 1 black balls. At each stepn=1, 2, .. , a ball is drawn at random from the urn andthen returned along with a new ball of the same color. LetRnandBnbe the numbers of redand black balls afternsteps, and let n=Rn/(Rn+Bn)be the fraction of red balls. Then nis amartingale relatve to the natural Functions and Markov , surely enough, MARTINGALES also arise inconnection with Markov chains; in fact, one of Doob s motivations in inventing them was toconnect the world of potential theory for Markov processes with the classical theory of sumsof independent random ,Y0,Y1.

8 Be a Markov chain on a denumerable statespaceYwith transition probability matrixP. A real-valued functionh:Y Ris calledhar-monicfor the transition probability matrixPif(11)Ph=h,equivalently, if for everyx Y,(12)h(x)= y Yp(x,y)h(y)=Exh(Y1).HereExdenotes the EXPECTATION corresponding to the probability measurePxunder whichPx{Y0=x}=1. Notice the similarity between equation (12) and the equation for the stationarydistribution one is just thetransposeof the h is harmonic for the transition probability matrixPthen for every startingstate x Ythe sequence h(Yn)is a martingale under the probability measure his 800-page bookClassical Potential Theory and its Probabilistic Counterpartfor more on is once again nothing more than a routine calculation.

9 The key is the Markov prop-erty, which allows us to rewrite any CONDITIONAL EXPECTATION onY0,Fnas a CONDITIONAL expecta-tion onYn. Thus,E(h(Yn+1)|Y0,Fn)=E(h(Yn+1)|Yn)= y Yh(y)p(Yn,y)=h(Yn). from {Xn}n 0be a martingale relative to the sequenceY0,Y1, .. Let :R Rbe a convex function such thatE (Xn)< for eachn 0. Then thesequence {Zn}n 0defined by(13)Zn= (Xn)is asubmartingale. This is a consequence of the Jensen inequality and the martingale propertyof {Xn}n 0:E(Zn+1|Y0,Y1, .. ,Yn)=E( (Xn+1)|Y0,Y1, .. ,Yn) (E(Xn+1|Y0,Y1, .. ,Yn)= (Xn)=ZnUseful special cases: (a) (x)=x2, and (b) (x)=exp{ x}.2. MART I N GA L E AN DSU BMART I N GA L ETR AN S F O R MSAccording to the Merriam-Webster Collegiate Dictionary, amartingaleisany of several systems of betting in which a player increases the stake usually bydoubling each time a bet is use of the term in the theory of probability derives from the connection withfair gamesorfair bets; and the importance of the theoretical construct in the world of finance also derivesfrom the connection with fair bets.)

10 Seen in this light, the notion of amartingale transform,which we are about to introduce, becomes most natural. Informally, a martingale transform isnothing more than a system of placing bets on a fair formal definition of a martingale transform requires two aux-iliary notions:martingale differencesandpredictable sequences. LetX0,X1, .. be a martingalerelative to another sequenceY0,Y1, .. (or to a filtration {Fn}n 0). Forn=1, 2, .. , define(14) n=Xn Xn 1;to be the martingale difference sequence associated with the sequence Z1,Z2, .. relative to the filtrationFnis a sequence of random vari-ables such that for eachn=1, 2, .. the random variableZnis measurable relative toFn 1. Ingambling (and financial) contexts,Znmight represent the size (say, in dollars) of a bet paced onthenth play of a game, while nrepresents the ( random ) payoff of thenth play per dollar requirement that the sequenceZnbe predictable in such contexts is merely an assertionthat the gambler not be ,X1.