Probability Theory - University of Arizona

1 Probability TheoryDecember 12, 2006 Contents1 Probability Measures, Random Variables, and Measures and Probabilities .. Random Variables and Distributions .. Integration and Expectation .. 132 measure Sierpinski Class Theorem .. Finitely additive set functions and their extensions to measures .. 213 Multivariate Independence .. Fubini s theorem .. Transformations of Continuous Random Variables .. Conditional Expectation .. Normal Random Variables .. 394 Notions of Inequalities .. Modes of Convergence .. Uniform Integrability .. 475 Laws of Large Product Topology .. Daniell-Kolmogorov Extension Theorem .. Weak Laws of Large Numbers .. Strong Law of Large Numbers .. Applications .. Large Deviations .. 686 Convergence of Probability Prohorov Metric .. Weak Convergence .. Prohorov s Theorem .. Separating and Convergence Determining Sets.

2 Characteristic Functions .. 867 Central Limit The Classical Central Limit Theorem .. Infinitely Divisible Distributions .. Weak Convergence of Triangular Arrays .. Applications of the L evy-Khinchin Formula .. 9721 Probability Measures, Random Variables, and ExpectationA phenomena is calledrandomif the exact outcome is uncertain. The mathematical study of randomness iscalled thetheory of Probability model has two essential pieces of its , the sample space , the set of possible a collection ofoutcomes. and a subset of the sample spaceA . , the Probability assigns a number to each Measures and ProbabilitiesLet be a sample space { 1, .. , n}and forA , let|A|denote the number of elements inA. Then theprobability associated withequally likely eventsP(A) =|A|| |( )reports the fraction of outcomes in that are also facts are (A) IfA B= , thenP(A B) =P(A) +P(B). ( ) = these facts, we can derive several others:Exercise IfA1.

3 , Akare pairwise disjoint ormutually exclusive, (Ai Aj= ifi6=j.) thenP(A1 A2 Ak) =P(A1) +P(A2) + +P(Ak).2. For any two eventsAandB,P(A B) =P(A) +P(B) P(A B).3. IfA BthenP(A) P(B).4. For anyA,0 P(A) LettingAcdenote thecomplementofA, thenP(Ac) = 1 P(A).The abstracting of the idea of Probability beyond finite sample spaces and equally likely events beginswith demanding that the domain of the Probability have properties that allow for the operations in theexercise above. This leads to the following nonempty collectionAof subsets of a setSis called AimpliesAc , A2 AimpliesA1 A2 If, in addition,{An:n= 1,2, } Aimplies n=1An A, thenAis called a LetS=R, then show that the collection ki=1(ai, bi], ai< bi , k= 1,2, ..is an Let{Fi;i 1}be an increasing collection of -algebras, then i=1 Fiis an algebra. Give an exampleto show that it is not a can use these ideas we can begin with{An:n 1} Aand create other elements inA.)

4 For example,lim supn An= n=1 m=nAm={Aninfinitely often}={ },( )andlim infn An= n=1 m=nAm={Analmost always}={ }.( )Exercise why the termsinfinitely oftenandalmost alwaysare appropriate. Show that{ }={ }cDefinition -algebra, then pair(S,S)is called ameasurable arbitrary intersection of -algebras is a -algebra. The power set ofSis a any collection of subsets. Then, (C)will denote the smallest -algebra the exercise above, this is the (non-empty) intersection of all -algebras For a single setA, (A) ={ , A, Ac, S}.2. IfCis a -algebra, then (C) = IfS Rd, or, more generally,Sis a topological space , andCis the set of the open sets inS, then (C)is called theBorel -algebraand denotedB(S).4. Let{(Si,Si)1 i n}be a set of measurable spaces, then theproduct -algebraon the spaceS1 Snis (S1 Sn).These -algebras form the domains of (S,S)be a measurable space . A function :S [0, ]is called ameasureif1. ( ) = (Additivity) IfA B= then (A B) = (A) + (B).

5 3. (Continuity) IfA1 A2 , andA= n=1An, then (A) = limn (An).If in addition,4. (Normalization) (S) = 1, is called 1 and 2 are needed ifSis an algebra. We need to introduce the notion of limit as in 3 to bring inthe tools of calculus and 3 is continuity from below. Show that measures have continuity from above. IfA1 A2 , andA= n=1An, then (A1)< implies (A) = limn (An).Give an example to show that the hypothesis (A1)< is triple(S,S, )is called ameasure spaceor aprobability spacein the case that isa will generally use the triple ( ,F, P) for a Probability space . An element in is called anoutcome,asample pointorrealizationand a member ofFis called that property 3 can be replaced with:3 . (Countable additivity) If{An;n 1}are pairwise disjoint (i6=jimpliesAi Aj= ), then ( n=1An) = n=1 (An).Exercise {A N; (A) = limn |A {1,2, .. , n}|nexists.}.Definition measure is called -finiteif can we can find{An;n 1} S, so thatS= n=1 Anand (An)< for (first two Bonferoni inequalities) Let{An:n 1} S.

6 ThenP(n j=1Aj) n j=1P(Aj)( )andP(n j=1Aj) n j=1P(Aj) 1 i<j nP(Ai Aj).( )Example (Counting measure , ) ForA S, (A)is the number of elements inA. Thus, (A) = ifAhas infinitely many elements. is not -finite if is (Lebesgue measuremon(R1,B(R1)) For the open interval(a, b), setm(a, b) =b a. Lebesgue measuregeneralizes the notion of length. There is a maximum -algebra which is smaller than the power set inwhich this measure can be defined. Lebesgue measure restricted to the set[0,1]is a Probability (Product measure ) Let{(Si,Si, i); 1 i k}bek -finite measure spaces. Then the product measure 1 kis the unique measure on (S1 Sn)such that 1 k(A1 Ak) = 1(A1) k(Ak)for allAi Si, i= 1, .. measure onRkis the product measure ofkcopies of Lebesgue measure eventsA1 Akare calledmeasurable rectangles. We shall learn soon why a measure is determinedby its value on measurable sayAoccurs almost everywhere( ) if (Ac) = 0.)

7 If is a Probability , we sayAoccurs almost surely( ). If two functionsfandgsatsifyf= , then we say thatgis Random Variables and DistributionsDefinition : (S,S) (T,T)be a function between measure spaces, thenfis called measurableiff 1(B) Sfor everyB T.( )If(S,S)has a Probability measure , thenfis called a random random variables we often write{X B}={ :X( ) B}=X 1(B).Generally speaking, weshall use capital letters near the end of the alphabet, , Y, Zfor random variables. The range ofXiscalled thestate often called arandom vectorif the state space is a Cartesian The composition of measurable functions is IfX: ( ,F) (S,S)is a random variable, then the collection (X) ={X 1(B) :B S}( )is a -algebra in . Thus,Xis a random variable if and only if (X) The collection{B S:X 1(B) F}is a -algebra inS4. IfSandTare topological spaces andSandTare their respective Borel -algebras, then any continuousfunctionf: (S,S) (T,T)is would like to limit the number of events{X B}we need to verify are inFto establish thatXisa random variable.

8 Here is an example in the case of : [ , ]. ThenXis a random variable if and only ifX 1([ , x]) ={X x} F( )for everyx a random variable, then obviously the condition the exercise above,C={B [ , ] :X 1(B) F}is a -algebra. Thus, we need only show that it contains the open sets. Because an open subset of [ , ]is the countable collection of open intervals, it suffices to show that this collectionCcontains sets of the form[ , x1),(x2, x1),and (x2, ].However, the middle is the intersection of the first and third and the third is the complement of [ , x2]whose iverse image is inFby assumption. Thus, we need only show thatCcontains sets of the form[ , x1).However, if we choosesn< x1with limn sn=x1, then[ , x1) = n=1[ , sn] = n=1(sn, ] {Xn;n 1}is a sequence of random variables, thenX= lim supn Xnis a random LetAbe an event. Theindicator functionforA,IA(s)equals 1 ifs A, and 0 iss6 A simple functionetake on a finite number of distinct values,e(s) = ni=1aiIAi(s),A1, , An S,anda1, , an S.]

9 Thus,Ai={s:e(s) =ai}. Call this class of a countable collection of sets,{An:n 1}s lim infn Anif and only iflim infn IAn(s) = lim supn Anif and only iflim supn IAn(s) = a sequence of sets,{An:n 1}, if there exists a setAso thatIA= limn IAn,we writeA= limn two sets,AandB, define thesymmetric differenceA B= (A\B) (B\A). Let{An:n 1}and{Bn:n 1}be sequences of sets with a limit. Show (An Bn) = (limn An) (limn Bn). (An Bn) = (limn An) (limn Bn). (An\Bn) = (limn An)\(limn Bn). (An Bn) = (limn An) (limn Bn).Definition any random variableX: S, thedistributionofXis the Probability measure (B) =P(X 1(B)) =P{X B}.( )Exercise is a Probability measure : R, then thedistribution functionis given byFX(x) =P{X x}= ( , x].Theorem distribution function has the following FX(x) = 1,limx FX(x) = right SetFX(x ) = limp x FX(p). ThenFX(x ) =P{X < x}. {X=x}=F(x) F(x ). we are determining limits in a metric space , checking the limits for sequences is Use the fact thatx1 x2implies that{X x1} {X x2}.)

10 2. Let{sn;n 1}be an increasing sequence with limit , Then{X s1} {X s2} , and n=1{X sn}= . If{rn;n 1}is a decreasing sequence with limit , Then{X r1} {X r2} , and n=1{X rn}= . Now, use the continuity properties of a Now, let{rn;n 1}be a decreasing sequence with limitx. Then{X r1} {X r2} , and n=1{X rn}={X x}. Again, use the continuity properties of a Also, if{sn;n 1}is a strictly increasing sequence with limitx, Then{X s1} {X s2} ,and n=1{X sn}={X < x}. Once more, use the continuity properties of a Note thatP{X=x}+P{X < x}=P{X x}and use 3 and , we have the 1, 2 and 3 above, then it is the distribution of some random ( ,F, P) = ((0,1),B((0,1)), m) wheremis Lebesgue measure and define for each ,X( ) = sup{ x:F( x)< }.Note that becauseFis nondecreasing{ x:F( x)< }is an interval that isnotbounded { :X( ) x}={ : F(x)}.BecausePis Lebesgue measure on (0,1), the claim shows thatP{ :X( ) x}=P{ : F(x)}=F(x).