Conditional Expectation - Department of Mathematics ...

1 Lecture10: ConditionalExpectation1of17 Course:Theory of probability ITerm:Fall2013 Instructor:Gordan ZitkovicLecture10 ConditionalExpectationThe definition and existence of Conditional expectationFor eventsA,BwithP[B]>0, we recall the familiar objectP[A|B] =P[A B]P[B].We say thatP[A|B]theconditional probability ofA, isimportant to note that the conditionP[B]>0 is crucial. WhenXandYare random variables defined on the same probability space, we oftenwant to give a meaning to the expressionP[X A|Y=y], even thoughit is usually the case thatP[Y=y] =0. When the random vector(X,Y)admits a joint densityfX,Y(x,y), andfY(y)>0, the conceptof Conditional densityfX|Y=y(x) =fX,Y(x,y)/fY(y)is introduced andthe quantityP[X A|Y=y]is given meaning via AfX|Y=y(x,y) this procedure works well in the restrictive case of absolutelycontinuous random vectors, we will see how it is encompassed bya general concept of a Conditional Expectation .

2 Since probability issimply an Expectation of an indicator, and expectations are linear, itwill be easier to work with expectations and no generality will be main conceptual leaps here are:1) we condition with respectto a -algebra, and2) we view the Conditional Expectation itself as arandom variable. Before we illustrate the concept in discrete time, hereis the a sub- -algebra ofF, and letX L1be arandom variable. We say that the random variable is (a version of)theconditional Expectation ofXwith respect toG- and denote it byE[X|G]- if1. isG-measurable, [ 1A] =E[X1A], for allA Updated:January24,2015 Lecture10: that( ,F,P)is a probability space where ={a,b,c,d,e,f},F=2 andPis uniform.

3 LetX,YandZberandom variables given by (in the obvious notation)E[X(Y)]XE[X( )]XX (abcdef1 3 3557),Y (abcdef2 2 1177)andZ (abcdef3 3 3322)We would like to think aboutE[X|G]as the average ofX( )over all which are consistent with the our current information (which isG). Forexample, ifG= (Y), then the information contained inGis exactlythe information about the exact value ofY. Knowledge of the fact thatY=ydoes not necessarily reveal the true , but certainly rules outall those for whichY( )6= our specific case, if we know thatY=2, then =aor =b,and the expected value ofX, given thatY=2, is12X(a) +12X(b) = , this average equals 4 forY=1, and 6 forY=7.

4 Let us showthat the random variable defined by this average, , (abcdef2 2 4466),satisfies the definition ofE[X| (Y)], as given above. The integrabilityis not an issue (we are on a finite probability space), and it is clearthat is measurable with respect to (Y). Indeed, the atoms of (Y)are{a,b},{c,d}and{e,f}, and is constant over each one of , we need to check thatE[ 1A] =E[X1A], for allA (Y),which for an atomAtranslates into ( ) =1P[A]E[X1A] = AX( )P[{ }|A], for all moral of the story is that whenAis an atom, part3. of into a requirement that be constant onAwithvalue equal to the Expectation ofXoverAwith respect to the condi-tional probabilityP[ |A].

5 In the general case, when there are no atoms,3. still makes sense and conveys the same , since the atoms of (Z)are{a,b,c,d}and{e,f}, it is clearthatE[X| (Z)]( ) = 3, {a,b,c,d},6, {e,f}.Last Updated:January24,2015 Lecture10: ConditionalExpectation3of17 Look at the illustrations above and convince yourself thatE[E[X| (Y)]| (Z)] =E[X| (Z)].A general result along the same lines - called thetower property of con-ditional Expectation - will be stated and proved first task is to prove that Conditional expectations always is finite (as explained above) or countable, we can alwaysconstruct them by averaging over atoms.

6 In the general case, a differentargument is needed. In fact, here are a sub- -algebraGofF. Then1. there exists a Conditional expectationE[X|G]for any X L1, and2. any two Conditional expectations of X L1are (Uniqueness): Suppose that and both satisfy1.,2. and3. ThenE[ 1A] =E[ 1A], for allA { 1n}, we haveAn Gand soE[ 1An] =E[ 1An] E[( +1n)1An] =E[ 1An] +1nP[An].Consequently,P[An] =0, for alln N, so thatP[ > ] =0. By asymmetric argument, we also haveP[ < ] =0.(Existence): By linearity, it will be enough to prove that the condi-tional Expectation exists forX L1+.

7 Radon-Nikodym argument. Suppose, first, thatX 0 andE[X] =1, as the general case follows by additivity and scaling. Thenthe prescriptionQ[A] =E[X1A],defines a probability measure on( ,F), which is absolutely continu-ous with respect toP. LetQGbe the restriction ofQtoG; it is triviallyabsolutely continuous with respect to the restrictionPGofPtoG. TheRadon-Nikodym theorem - applied to the measure space( ,G,PG)and the measureQG PG- guarantees the existence of the Radon-Nikodym derivative =dQGdPG L1+( ,G,PG).ForA G, we thus haveE[X1A] =Q[A] =QG[A] =EPG[ 1A] =E[ 1A].

8 Last Updated:January24,2015 Lecture10: ConditionalExpectation4of17where the last equality follows from the fact that , is (a version of) the Conditional expectationE[X|G]. Suppose, first, thatX L2. LetHbe the familyof allG-measurable elements inL2. Let Hdenote the closure ofHinthe topology induced byL2-convergence. Being a closed and convex(why?) subset ofL2, Hsatisfies all the conditions of Problem??so thatthere exists Hat the minimalL2-distance fromX(whenX H,we take =X). The same problem states that has the followingproperty:E[( )(X )] 0 for all H,and, since His a linear space, we haveE[( )(X )] =0, for all remains to pick of the form = +1A H,A G, to concludethatE[X1A] =E[ 1A], for allA next step is to show that isG-measurable (after a modifi-cation on a null set, perhaps).

9 Since H, there exists a sequence{ n}n Nsuch that n inL2. By Corollary??, , forsome subsequence{ nk}k Nof{ n}n N. Set =lim infk N nk L0([ , ],G)and = 1{| |< }, so that = , , and still need to remove the restrictionX L2+. We start witha generalX L1+and defineXn=min(X,n) L + L2+. Let n=E[Xn|G], and note thatE[ n+11A] =E[Xn+11A] E[Xn1A] =E[ n1A]. It follows (just like in the proof of uniqueness above) that n n+1, We define =supn n, so that n , Then, forA G, the monotone-convergence theorem implies thatE[X1A] =limnE[Xn1A] =limnE[ n1A] =E[ 1A],and it is easy to check that 1{ < } L1(G)is a version ofE[X|G].

10 Is no canonical way to choose the version of theconditional Expectation . We follow the convention started with Radon-Nikodym derivatives, and interpret a statement such at E[X|G], , to mean that , , for any version of the conditionalexpectation ofXwith respect we use the symbolL1to denote the set of all of random variables inL1, we can write:E[ |G]:L1(F) L1(G),Last Updated:January24,2015 Lecture10: ConditionalExpectation5of17butL1(G)canno t be replaced byL1(G)in a natural way. SinceX=X , , implies thatE[X|G] =E[X |G], (why?), we consider condi-tional Expectation as a map fromL1(F)toL1(G)E[ |G]:L1(F) L1(G).