Example: confidence

Lecture 10 - University of Texas at Austin

Lecture10: ConditionalExpectation1of17 Course:Theory of Probability ITerm:Fall2013 Instructor:Gordan ZitkovicLecture10 ConditionalExpectationThe definition and existence of conditional expectationFor eventsA,BwithP[B]>0, we recall the familiar objectP[A|B] =P[A B]P[B].We say thatP[A|B]theconditional probability ofA, isimportant to note that the conditionP[B]>0 is crucial. WhenXandYare random variables defined on the same probability space, we oftenwant to give a meaning to the expressionP[X A|Y=y], even thoughit is usually the case thatP[Y=y] =0. When the random vector (X,Y)admits a joint densityfX,Y(x,y), andfY(y)>0, the conceptof conditional densityfX|Y=y(x) =fX,Y(x,y)/fY(y)is introduced andthe quantityP[X A|Y=y]is given meaning via AfX|Y=y(x,y) this procedure works well in the restrictive case of absolutelycontinuous random vectors, we will see how it is encompassed bya general concept of a conditional expectation.

Jan 24, 2015 · When the random vector (X,Y) admits a joint density fX,Y(x,y), and fY(y) > 0, the concept of conditional density f XjY=y(x) = f, Y(x,y)/f (y) is introduced and the quantity P[X 2AjY = y] is given meaning via R A f XjY=y(x,y)dx. While this procedure works well in the restrictive case of absolutely continuous random vectors, we will see how it is ...

Tags:

  Vector, Random, Random vectors

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Lecture 10 - University of Texas at Austin

1 Lecture10: ConditionalExpectation1of17 Course:Theory of Probability ITerm:Fall2013 Instructor:Gordan ZitkovicLecture10 ConditionalExpectationThe definition and existence of conditional expectationFor eventsA,BwithP[B]>0, we recall the familiar objectP[A|B] =P[A B]P[B].We say thatP[A|B]theconditional probability ofA, isimportant to note that the conditionP[B]>0 is crucial. WhenXandYare random variables defined on the same probability space, we oftenwant to give a meaning to the expressionP[X A|Y=y], even thoughit is usually the case thatP[Y=y] =0. When the random vector (X,Y)admits a joint densityfX,Y(x,y), andfY(y)>0, the conceptof conditional densityfX|Y=y(x) =fX,Y(x,y)/fY(y)is introduced andthe quantityP[X A|Y=y]is given meaning via AfX|Y=y(x,y) this procedure works well in the restrictive case of absolutelycontinuous random vectors, we will see how it is encompassed bya general concept of a conditional expectation.

2 Since probability issimply an expectation of an indicator, and expectations are linear, itwill be easier to work with expectations and no generality will be main conceptual leaps here are:1) we condition with respectto a -algebra, and2) we view the conditional expectation itself as arandom variable. Before we illustrate the concept in discrete time, hereis the a sub- -algebra ofF, and letX L1be arandom variable. We say that the random variable is (a version of)theconditional expectation ofXwith respect toG- and denote it byE[X|G]- if1. isG-measurable, [ 1A] =E[X1A], for allA Updated:January24,2015 Lecture10: that( ,F,P)is a probability space where ={a,b,c,d,e,f},F=2 andPis uniform. LetX,YandZberandom variables given by (in the obvious notation)E[X(Y)]XE[X( )]XX (abcdef1 3 3557),Y (abcdef2 2 1177)andZ (abcdef3 3 3322)We would like to think aboutE[X|G]as the average ofX( )over all which are consistent with the our current information (which isG).

3 Forexample, ifG= (Y), then the information contained inGis exactlythe information about the exact value ofY. Knowledge of the fact thatY=ydoes not necessarily reveal the true , but certainly rules outall those for whichY( )6= our specific case, if we know thatY=2, then =aor =b,and the expected value ofX, given thatY=2, is12X(a) +12X(b) = , this average equals 4 forY=1, and 6 forY=7. Let us showthat the random variable defined by this average, , (abcdef2 2 4466),satisfies the definition ofE[X| (Y)], as given above. The integrabilityis not an issue (we are on a finite probability space), and it is clearthat is measurable with respect to (Y). Indeed, the atoms of (Y)are{a,b},{c,d}and{e,f}, and is constant over each one of , we need to check thatE[ 1A] =E[X1A], for allA (Y),which for an atomAtranslates into ( ) =1P[A]E[X1A] = AX( )P[{ }|A], for all moral of the story is that whenAis an atom, part3.

4 Of into a requirement that be constant onAwithvalue equal to the expectation ofXoverAwith respect to the condi-tional probabilityP[ |A]. In the general case, when there are no atoms,3. still makes sense and conveys the same , since the atoms of (Z)are{a,b,c,d}and{e,f}, it is clearthatE[X| (Z)]( ) = 3, {a,b,c,d},6, {e,f}.Last Updated:January24,2015 Lecture10: ConditionalExpectation3of17 Look at the illustrations above and convince yourself thatE[E[X| (Y)]| (Z)] =E[X| (Z)].A general result along the same lines - called thetower property of con-ditional expectation- will be stated and proved first task is to prove that conditional expectations always is finite (as explained above) or countable, we can alwaysconstruct them by averaging over atoms. In the general case, a differentargument is needed.

5 In fact, here are a sub- -algebraGofF. Then1. there exists a conditional expectationE[X|G]for any X L1, and2. any two conditional expectations of X L1are (Uniqueness): Suppose that and both satisfy1.,2. and3. ThenE[ 1A] =E[ 1A], for allA { 1n}, we haveAn Gand soE[ 1An] =E[ 1An] E[( +1n)1An] =E[ 1An] +1nP[An].Consequently,P[An] =0, for alln N, so thatP[ > ] =0. By asymmetric argument, we also haveP[ < ] =0.(Existence): By linearity, it will be enough to prove that the condi-tional expectation exists forX L1+. Radon-Nikodym argument. Suppose, first, thatX 0 andE[X] =1, as the general case follows by additivity and scaling. Thenthe prescriptionQ[A] =E[X1A],defines a probability measure on( ,F), which is absolutely continu-ous with respect toP. LetQGbe the restriction ofQtoG; it is triviallyabsolutely continuous with respect to the restrictionPGofPtoG.

6 TheRadon-Nikodym theorem - applied to the measure space( ,G,PG)and the measureQG PG- guarantees the existence of the Radon-Nikodym derivative =dQGdPG L1+( ,G,PG).ForA G, we thus haveE[X1A] =Q[A] =QG[A] =EPG[ 1A] =E[ 1A].Last Updated:January24,2015 Lecture10: ConditionalExpectation4of17where the last equality follows from the fact that , is (a version of) the conditional expectationE[X|G]. Suppose, first, thatX L2. LetHbe the familyof allG-measurable elements inL2. Let Hdenote the closure ofHinthe topology induced byL2-convergence. Being a closed and convex(why?) subset ofL2, Hsatisfies all the conditions of Problem??so thatthere exists Hat the minimalL2-distance fromX(whenX H,we take =X). The same problem states that has the followingproperty:E[( )(X )] 0 for all H,and, since His a linear space, we haveE[( )(X )] =0, for all remains to pick of the form = +1A H,A G, to concludethatE[X1A] =E[ 1A], for allA next step is to show that isG-measurable (after a modifi-cation on a null set, perhaps).

7 Since H, there exists a sequence{ n}n Nsuch that n inL2. By Corollary??, , forsome subsequence{ nk}k Nof{ n}n N. Set =lim infk N nk L0([ , ],G)and = 1{| |< }, so that = , , and still need to remove the restrictionX L2+. We start witha generalX L1+and defineXn=min(X,n) L + L2+. Let n=E[Xn|G], and note thatE[ n+11A] =E[Xn+11A] E[Xn1A] =E[ n1A]. It follows (just like in the proof of uniqueness above) that n n+1, We define =supn n, so that n , Then, forA G, the monotone-convergence theorem implies thatE[X1A] =limnE[Xn1A] =limnE[ n1A] =E[ 1A],and it is easy to check that 1{ < } L1(G)is a version ofE[X|G]. is no canonical way to choose the version of theconditional expectation. We follow the convention started with Radon-Nikodym derivatives, and interpret a statement such at E[X|G], , to mean that , , for any version of the conditionalexpectation ofXwith respect we use the symbolL1to denote the set of all of random variables inL1, we can write:E[ |G]:L1(F) L1(G),Last Updated:January24,2015 Lecture10: ConditionalExpectation5of17butL1(G)canno t be replaced byL1(G)in a natural way.

8 SinceX=X , , implies thatE[X|G] =E[X |G], (why?), we consider condi-tional expectation as a map fromL1(F)toL1(G)E[ |G]:L1(F) L1(G).PropertiesConditional expectation inherits many of the properties from the or-dinary expectation. Here are some familiar and some new X,Y,{Xn}n Nbe random variables inL1, and letGandHbe sub- -algebras ofF. Then1.(linearity)E[ X+ Y|G] = E[X|G] + E[Y|G], (monotonicity)X Y, , impliesE[X|G] E[Y|G], (identity onL1(G))If X isG-measurable, then X=E[X|G], Inparticular, c=E[c|G], for any constant c (conditional Jensen s inequality)If :R Ris convex andE[| (X)|]< thenE[ (X)|G] (E[X|G]), (Lp-nonexpansivity)If X Lp, for p [1, ], thenE[X|G] Lpand||E[X|G]||Lp ||X|| particular,E[|X||G] |E[X|G]| (pulling out what s known)If Y isG-measurable and XY L1, thenE[XY|G] =YE[X|G], (L2-projection)If X L2, then =E[X|G]minimizesE[(X )2]over allG-measurable random variables (tower property)IfH G, thenE[E[X|G]|H] =E[X|H], (irrelevance of independent information)IfHis independent of (G, (X))thenE[X| (G,H)] =E[X|G], particular, if X is independent ofH, thenE[X|H] =E[X], Updated:January24,2015 Lecture10: ConditionalExpectation6of1710.

9 (conditional monotone-convergence theorem)If0 Xn Xn+1, , forall n Nand Xn X L1, , thenE[Xn|G] E[X|G], (conditional Fatou s lemma)If Xn 0, , for all n N, andlim infnXn L1, thenE[lim infnX|G] lim infnE[Xn|G], (conditional dominated-convergence theorem)If|Xn| Z, for all n Nand some Z L1, and if Xn X, , thenE[Xn|G] E[X|G], and :Some of the properties are provedin others are only com-mented upon, since they are either sim-ilar to the other ones or otherwise (linearity)E[( X+ Y)1A] =E[( E[X|G] + E[Y|G])1A], forA (monotonicity)UseA={E[X|G]>E[Y|G]} Gto obtain a contra-diction ifP[A]> (identity onL1(G))Check the (conditional Jensen s inequality)Use the result of Lemma??whichstates that (x) =supn N(an+bnx), where{an}n Nand{bn}n Nare sequences of real (Lp-nonexpansivity)Forp [1, ), apply conditional Jensen s in-equality with (x) =|x|p.]

10 The casep= follows (pulling out what s known)ForYG-measurable andXY L1, we needto show thatE[XY1A] =E[YE[X|G]1A], for allA G.( )Let us prove a seemingly less general statement:E[ZX] =E[ZE[X|G]], for allG-measurableZwithZX L1.( )The statement ( ) will follow from it by takingZ=Y1A. ForZ= nk=1 k1Ak, ( ) is a consequence of the definition of con-ditional expectation and linearity. Let us assume that bothZandXare nonnegative andZX L1. In that case we can find a non-decreasing sequence{Zn}n Nof non-negative simple random vari-ables withZn Z. ThenZnX L1for alln Nand the mono-tone convergence theorem implies thatE[ZX] =limnE[ZnX] =limnE[ZnE[X|G]] =E[ZE[X|G]].Last Updated:January24,2015 Lecture10: ConditionalExpectation7of17 Our next task is to relax the assumptionX L1+to the original oneX L1. In that case, theLp-nonexpansivity forp=1 implies that|E[X|G]| E[|X||G] ,and so|ZnE[X|G]| ZnE[|X||G] ZE[|X||G].


Related search queries