Probability Theory 1 Lecture Notes - pi.math.cornell.edu

1 Ersonaleducationaluseonlyandarenottob ,muchofthestructure,andsomeofthelanguage comesdirectlyfromthecoursetext, ' ' :FinishedSection1 Day2:FinishedSection2 Day3:Uptode nitionofsemialgebraDay4:FinishedSection3 Day5:ThroughreviewofintegrationDay6:Thro ughLimitTheoremsDay7 :FinishedSection6 Day10 :FinishedSection7 Day14 :FinishedSection8 Day16 :UptoClaimin3-seriesTheoremDay19:Finishe dSection10 Day20 :FinishedSection11 Day23 :FinishedSection13 Day28 :FinishedSection14 Day30:Throughb eginningofpro :FinishedSection16 Day34:ThroughProp :Startedpro (skipp edWald2)Day36 (skipp edChung-Fuchs)Day38 ( ,F,P)withP( ) = canb eanyset,anditcanb ethoughtofasthecollectionofallp ossibleoutcomesofsomeexp erimentorallp -algebra(or - eld)F 2 satis es1)Fisnonempty2)E F EC F3)Foranycountablecollection{Ei}i I F, i IEi ,andcanb eregardedassetsofelementaryoutcomesab erimenthaso ccurred(ortheobservationhasb eenmade),ameaningfulstatementab outE FisP(E).

2 Afterward,ameaningfulstatementiswhethero rnotEo :F [0,1]satis es1)P( ) = 12)foranycountabledisjointcollection{Ei} i I,P( i IEi)= i IP(Ei).TheinterpretationisthatP(A)repres entsthechancethateventAo ccurs(thoughthereisnogeneralconsensusab outwhatthatactuallymeans).Ifpissomeprop ertyandA={ :p( )istrue}issuchthatP(A) = 1,thenwesaythatpholdsalmostsurely, almosteverywhere ossibletohaveaneventE FwithE6= andP(E) = ,forinstance,thereisadistinctionb etween imp ossible and withprobabilityzero : ={1,2,3,4,5,6},F= 2 ,P(E) =|E| (p ossiblybiased)coin: ={H,T},F= 2 ={ ,{H},{T},{H,T}},Psatis esP({H}) =pandP({T}) = 1 pforsomep (0,1). ointintheunitinterval: = [0,1],F=B[0,1]=BorelSets,P=Leb erimenthereistopickarealnumb erb eaking,uniformitycorresp ondstotranslationinvariance,whichisthepr imaryde ningprop ertyofLeb [0,1]hasP({x}) = 0,sotheexp : =R,F=B,P(E) =1 2 Ee : =N {0},F= 2 ,P(E) =e k E kk!

3 ,probabilitywasde nedintermsofa nitenumb erofequallylikelyoutcomes( )sothat| |< ,F= 2 ,andP(E) =|E|| |.Whenthesamplespaceiscountablyin nite( ),or nitebuttheoutcomesarenotnecessarilyequal lylikely( ),onecansp eakofprobabilitiesintermsweightedoutcome sbytakingafunctionp: [0,1]with p( ) = 1andsettingP(E) = Ep( ).Formostpracticalpurp oses,thiscanb egeneralizedtothecasewhere Rbytakingaweightingfunctionf: [0, )with f(x)dx= 1andsettingP(E) = Ef(x)dx( ),butonemustb ecarefulsincetheintegralisnotde nedforallsetsE( *). ,resp ,f=dPdmistheRadon-Niko dymderivativeofPwithresp ecttoLeb esguemeasure, ,p=dPdcwhereciscountingmeasureon .Measuretheoryprovidesaunifyingframework inwhichtheseideascanb emaderigorous, ,notethatintheformalaxiomaticconstructio nofprobabilityasameasurespacewithtotalma ss1,thereisabsolutelynomentionofchanceor randomness,sowecanuseprobabilitywithoutw orryingab (S,G),wede nean(S,G)-valuedrandomvariabletob eameasurablefunctionX: ,theunquali edterm randomvariable willrefertothecase(S,G) = (R,B).]

4 WetypicallythinkofXasanobservable,oramea surementtob etakenaftertheexp erimenthasb eenp Fandde ningtheindicatorfunction,1A( ) ={1, A0, ( ,F,P)isaprobabilityspaceandXisan(S,G)-va luedrandomvariable,thenXinducesthepushfo rwardprobabilitymeasure =P X 1on(S,G).Frequently,wewillabusenotationa ndwriteP(X B) =P(X 1(B)) =P({ :X( ) B})for (B).Xalsoinducesthesub- -algebra (X) ={X 1(E) :E G} asthep ossibleoutcomesofanexp erimentandXasameasurementtob ep erformed,then (X) ,inprobabilityweareofteninterestedinvari oussub- -algebrasF0 F, ,iftheexp erimentisrollingasix-sideddie( ),thenF0={ ,{1,3,5},{2,4,6}, } (ormeanorexpectedvalue)ofareal-valuedran domvariableXon( ,F,P)isde nedasE[X] = X( )dP( )}

5 Whenevertheintegraliswell-de ectationisgenerallyinterpretedasaweighte daveragewhichgivesthe b estguess ectationsingreaterdetailso ,thep ointisthatmanyfamiliarob jectsfromundergraduateprobabilitycanb erigorouslyandsimplyde ,itshouldb ,itisequallyimp ortanttocultivateaprobabilisticwayofthin kingwherebyoneconceptualizesproblemsinte rmsofcointossing,cardshu ing,particletra jectories,andsoforth.*Anexampleofasubset of[0,1]whichhasnowell-de nedLeb esguemeasureisgivenbythefollowingconstru ction:De neanequivalencerelationon[0,1]byx yifandonlyifx y ,letE [0,1]consistofexactlyonep Q[0,1),de neEq=E+q(mod 1).ByconstructionEq Er= forr6=qand q Q[0,1)Eq= [0,1].]]

6 Thus,bycountableadditivity,wemusthave1 =m([0,1)) =m q Q[0,1)Eq = q Q[0,1)m(Eq).However,Leb esguemeasureistranslationinvariant,som(E q) =m(E) (Eq)isnotwell-de nedasm(Eq) = 0implies1 = 0andm(Eq)>0implies1 = .Theexistenceofnon-measurablesetscanb eprovedusingslightlyweakerassumptionstha ntheaxiomofchoice(suchastheBo oleanprimeidealtheorem),butithasb ,theBanach-TarskiparadoxshowsthatinZFC,t hereisno nitelyadditivemeasurede nedonallsubsetsofEuclideanspacewhichisin variantundertranslationandrotation.(Thep aradoxisthatonecancutaunitballinto vepiecesandreassemblethemusingonlyrigidm otionstoobtaintwodisjointunitballs.) oint,weneedtoestablishsomefundamentalfac tsab outprobabilitymeasuresand ( ,F).]]]

7 (i)ComplementsForanyA F,P(AC) = 1 P(A).(ii)MonotonicityForanyA,B FwithA B,P(A) P(B).(iii)SubadditivityForanycountableco llection{Ei} i=1 F,P( i=1Ei) i=1P(Ei).(iv)Continuityfromb elowIfAi A( A2 ..and i=1Ai=A),thenlimn P(An) =P(A).(v)Continuityfromab oveIfAi A= i=1Ai,thenlimn P(An) =P(A). (i),1 =P( ) =P(AtAC) =P(A) +P(AC) (ii),P(B) =P(At(B\A)) =P(A) +P(B\A) P(A).For(iii),we disjointify thesetsbyde ningF1=E1andFi=Ei\( i 1j=1Ej)fori >1,andobservethattheF isaredisjointand ni=1Fi= ni=1 Eiforalln N { }.SinceFi Eiforalli,wehaveP( i=1Ei)=P( i=1Fi)= i=1P(Fi) i=1P(Ei).For(iv),setB1=A1andBi=Ai\Ai 1fori >1,andnotethattheB isaredisjointwith ni=1Bi=Anand i=1Bi= (A) =P( i=1Bi)= i=1P(Bi) = limn n i=1P(Bi)= limn P(n i=1Bi)= limn P(An).

8 For(v),ifA1 A2 ..andA= i=1Ai,thenAC1 AC2 ..andAC= ( i=1Ai)C= i=1 ACi,soitfollowsfrom(i)and(iv)thatP(A) = 1 P(AC) = 1 limn P(ACn) = limn (1 P(ACn))= limn P(An). Notethat(ii)-(iv)holdforanymeasurespace( S,G, ),(v)istrueforarbitrarymeasurespacesunde rtheassumptionthatthereissomeAiwith (Ai)< ,and(i)holdsforall nitemeasuresup onreplacing1with (S). out rstobservationisProp ,if{Fi}i Iisacol lectionof -algebrason ,thensois i 2 ,thereisasmallest -algebracontainingA-namely,theintersecti onofall -algebrageneratedbyAandisdenotedby (A).NotethatifFisa -algebraandA F,then (A) ortantclassofexamplesaretheBorel -algebras:If(X,T)isatop ologicalspace,thenBX= (T)iscalledtheBorel ologyonR,theBorelsetsaregeneratedbyop enintervals,closedintervals,half-op enintervals,op enrays,andclosedrays,resp out -algebrasisDynkin's - Theorem, ,wewillneedthefollowingde 2 iscalleda 2 iscalleda -systemif(1) L(2)IfA,B LandA B,thenB\A L(3)IfAn LwithAn A,thenA (Dynkin).

9 IfPisa -systemandLisa -systemwithP L,then (P) eginbyobservingthattheintersectionofanyn umb erof -systemsisa -system,soforanycollectionA,thereisasmal lest -system`(A) cetoshowa)`(P)isa -algebra(sincethen (P) `(P) L).Infact,asoneeasilychecksthata -systemwhichisclosedunderintersectionisa -algebra(AC= \A,A B= (AC BC)C,and ni=1Ai i=1Ai),weneedonlytodemonstrateb)`(P) ,de neGA={B:A B `(P)} of,wewill rstshowc)GAisa -systemforeachA `(P),andthenprovethatb)followsfromc).Toe stablishc),letAb eanarbitrarymemb erof`(P).ThenA= A `(P),soGA3 .Also,foranyB,C GAwithB C,wehaveA (C\B) = (A C)\(A B) `(P)sinceA B,A C `(P)and`(P)isa -system,henceGAisclosedundersubsetdi ,foranysequence{Bn}inGAwithBn B,wehave(A Bn) (A B) `(P),soGAisclosedundercountableincreasin gunionsaswellandthusisa )impliesb).

10 Toseethatthisisthecase, rstnotethatsincePisa -system,P GAforeveryA P,soitfollowsfromc)that`(P) GAforeveryA ,foranyA P,B `(P),wehaveA B `(P).InterchangingAandByieldsA `(P)andB PimpliesA B `(P).ButthismeansifA `(P),thenP GA,andthusc)impliesthat`(P) ,itfollowsfromthede nitionofGAthatforanyA,B `(P),A B `(P). Itisnotesp eciallyimp ortanttocommitthedetailsofthispro oftomemory,butitworthseeingonceandyousho uldde on rstencounter, ,weshowthataprop ertyholdsona -systemthatweknowgeneratesthe ertyholdsisa -systeminordertoconcludethattheprop ertyholdsontheentire -systems, -systems,and -algebras,theintersectionofmonotoneclass esisamonotoneclass,soitmakessensetotalka b (MonotoneClassLemma).