Transcription of Lecture 10 : Conditional Expectation
1 Lecture10 : ConditionalExpectationSTAT205 Lecturer:JimPitmanScribe:CharlessC. nitionof ConditionalExpectationRecallthe\undergra duate"de nitionof conditionalprobability associatedwithBayes'RuleP(AjB) P(A; B)P(B)For a discreterandomvariableXwe haveP(A) =XxP(A; X=x) =XxP(AjX=x)P(X=x)andtheresultingformulaf orconditionalexpectationE(YjX=x)=Z Y(!)P(dwjX=x)=RX=xY(!)P(dw)P(X=x)=E(Y1(X =x))P(X=x)We wouldlike to extendthisto handlemoregeneralsituationswheredensitie sdon'texistor wewant to conditiononvery\complicated" randomvariableYwithEjYj<1onthespace( ;F;P)andsomesub- - eldG Fwewill de netheconditionalexpectationas thealmostsurelyuniquerandomvariableE(YjG )whichsatis (YjG) (Y Z) =E(E(YjG)Z)forallZwhichare bounded andG-measurableForG= (X) whenXis a discretevariable,thespace is simplypartitionedinto disjoint sets =tGn.
2 Ourde nitionforthediscretecasegivesE(Yj (X))=E(YjX)=XnE(Y1X=xn)P(X=xn)1X=xn=XnE( Y1Gn)P(Gn)1 Gnwhich is : escondition2 of De (Hint:showthattheconditionis satis ed forrandomvariablesof theformZ=1 GwhereG2 Cis a collectionclosed underintersectionandG= (C)theninvokeDynkin's ) WellDe (XjG)is uniqueup to almostsure Sketch:Supposethatbothrandomvariables^Ya nd^^Ysatisfyourconditionsforbeingthecond itionalexpectationE(YjX). LetW=^Y ^^Y. ThenWisG-measurableandE(W Z) = 0 forallZwhich we letZ=1W > (which is boundedandmeasurable)then P(W> ) E(W1W > ) = 0forall >0.
3 A similarargument appliedtoP(W< ) allowsus to concludethatP(jWj> ) = 0holdsforall andhenceW= 0 almostsurelymakingE(YjX) (XjG)existsWe've shownthatE(YjG) existsin thediscretecaseby writingoutanexplicitformulasothat\E(YjX) tointegrateslikeYoverG-measurablesets."W e give threedi erent \HandsOn"ProofThe rstis a handsonapproach by willmake 'sTower PropertySupposeG H Fare nested - eldsandE( jG)andE( jH)are bothwell de ned thenE(E(YjH)jG) =E(YjG) =E(E(YjG)jH)A specialcaseis whenG=f;; gthenE(YjG) =EYis a constant so it'seasyto seeE(E(YjH)jG) =E(E(Y)jH) =E(Y) andE(E(YjG)jH) =E(E(Y)jH) =E(Y)Proof Sketch:ExistenceviaLimitsFor a disjoint partitiontGi= andG2 G= (fGig)de neE(YjG) =XiE(Y1Gi)P(Gi)1 Giwherewe dealappropriatelywiththenigglingpossibil ity ofP(Gi) = 0 by eitherthrowingouttheo endingsetsor de ning00= now consideran arbitrarybutcountablygenerated - eldG.
4 Thissituationis nottoo restrictive,forexamplethe - eldassociatedwithanR-valuedrandomvariabl eXis generatedby thecountablecollectionfBi= (X ri) :r2Qg. If we setGn= (B1; B2; : : : ; Bn) thenGnis increasingto thelimitG1 G2 : : : G= ([Gn). For a givenntherandomvariableYn=E(YjGn) existsby ourexplicitde nitionabove sincewe candecomposethegeneratingsetinto a disjoint partitionof : ConditionalExpectation10-3 Now we show thatYnconvergesin someappropriatemannerto aY1which willthenfunctionas aversionofE(YjG). We willassumethatEjYj2<1 WriteYn=E(YjGn) =Y1+ (Y2 Y1) + (Y3 Y2) +: : :+ (Yn Yn 1).]
5 Thetermsin thissummationareorthogonalinL2so we cancomputethevarianceass2n=E(Y2n) =E(Y21) +E((Y2 Y1)2): : :+E((Yn Yn 1)2) (Y2) =E(Yn+ (Y Yn))<1. Thens2n"s21 s2< > mwe know againby orthogonality thatE((Yn Ym)2) =s2n s2m!0 asm!1sinces2nis justa Cauchy inL2andinvokingthecompletenessofL2we concludethatYn! tocheck thatY1is a satis esrequirement (1)sinceas a limitofG-measurablevariablesit check (2)we needto show thatE(Y G) =E(Y1G) forallGwhich ,it su cesto check fora much smallersetf1Ai:Ai2 AgwhereAis anintersectionclosedcollectionand (A) =G.
6 Takethiscollectionto beA=[ (Y Gm) =E(YmGm) =E(YnGm)holdsby thetower property foranyn > m. NotingthatE(YnZ)!E(Y1Z) is trueforallZ2L2by thecontinuity of innerproductthissequencemustgo to thedesiredlimitwhich givesE(YGm) =E(Y1Gm) constraintonG. (Hint:Bea bitmore clever: : :forY2L2look atE(YjG)forG FwithG abovesupGE(E(YjG)2) EY2so wecan chooseGnwithE(E(YjGn)2)increasingto nested butarguethatCn= (G1[G2[: : :[Gn)are andlet^Y= limnE(YjCn))). (Hint:ConsiderY 0andshowconvergence ofE(Y^nj G)thenturncrankonthestandard machinery) pulloutsomepower [2]( )If and are non-negative - nitemea-sures ona collectionGand (G) = 0 =) (G) = 0(written << , pronounced " is absolutelycontinuouswithrespect to ") forallG2 Gthenthere existsa non-negativeGmeasurablefunction^Ysuchtha t (G) =ZG^Y d Sketch.]]]]
7 ExistenceviaLebesgue-Radon-NikodymAssume Y 0 andde netheprob-ability measureQ(C) =ZCY dP=EY1 Cwhich is non-negative and nitebecauseEjYj<1andQis absolutelycontinuouswithrespecttoP. LRNimpliestheexistenceof^Ywhich satis esourrequirements to be a versionof theconditionalexpectation^Y=E(YjG). For generalYwe canemployE(Y+jG) E(Y jG).Lecture10: a ,closed, convexsetE in a Hilbertspace H containsa uniqueelementof Projectionsin HilbertSpaceGivena closed subspaceKof aHilbertspaceHandelementx2H, there existsa decompositionx=y+zwherey2 Kandz2K?
8 (theorthogonalcomplement).Theideaforthee xistenceof projectionsis to letybe theelement of smallestnorminx+Kandz=x y. See[2]( )fora fulldiscussionof Sketch: ExistenceviaHilbertSpaceProjectionSuppos eY2L2(F) andX2L2(G).Requirement (2)demandsthatforallXE((Y E(YjG))X) = 0which hasthegeometricinterpretationof requiringY E(YjG) to be orthogonalto thesubspaceL2(G). Requirement (1)says thatE(YjG)2L2(G) soE(YjG) is justtheorthogonalprojectionofYonto theclosedsubspaceL2(G). Thelemmaabove showsthatsuch a projectionis well de ConditionalExpectationIt'shelpfulto thinkofE( jG) as isolatesomeusefulpropertiesof conditionalexpectationwhich thereaderwillnodoubtwant toprove beforebelieving E( jG) is positive:Y 0!
9 E(YjG) 0) E( jG) is linear:E(aX+bYjG) =aE(XjG) +bE(YjG) E( jG) is a projection:E(E(XjG)jG) =E(XjG) Moregenerally, the\tower property".IfH GthenE(E(XjG)jH) =E(E(XjH)G) =E(XjH)Lecture10: ConditionalExpectation10-5 E( jG) commuteswithmultiplicationbyG-measurable variables:E(XYjG) =E(XjG)YforEjXYj<1andY2G E( jG) respectsmonotoneconvergence:0 Xn"X=)E(XnjG)"E(XjG) If is convex andEj (X)j<1thena conditionalformof Jensen'sinequality holds: (E(XjG) E( (X)jG) E( jG) is a continuouscontractionofLpforp 1:kE(XjG)kp kXkpandXnL2 !
10 XimpliesE(XnjG)L2 !E(XjG) G1 : : :,G1= ([Gi), andX2 Lpwithp 1 thenE(XjGn)a:s: !E(XjG1)E(XjGn)Lp !E(XjG1) : ( ;F)!(S;S)andsub- - eldG Fwede netheMarkov kernelQ(!; A) : S ![0;1]as a (carefully chosen)versionof theconditionalprobabilityP(X2 AjG)whichhastheproperties1.!7!Q(!; A)is a (G-measurable)versionofP(X2 AjG)for xed choice !Q(!; A)is a probabilitymeasure on(S;S)WhenS= andXis theidentitymapwecallQaregularconditional probabilityForG2 Gwe have thatP(X2A; G) =E(P(X2 AjG)1G) =ZGQ(!; A)P(d!)andin thecasewhenG= (Y) thekerneltakes theformQ(!)]
