Transcription of Probability Theory: STAT310/MATH230 April15,2021
1 Probability theory : STAT310/MATH230 December 15, 2020 Amir DemboE-mail of Mathematics, Stanford University, Stanford, CA 1. Probability , measure and Probability spaces, measures and Random variables and their Integration and the (mathematical) Independence and product measures54 Chapter 2. Asymptotics: the law of large Weak laws of large The Borel-Cantelli Strong law of large numbers85 Chapter 3. Weak convergence,cltand Poisson The Central Limit Weak Characteristic Poisson approximation and the Poisson Random vectors and the multivariateclt141 Chapter 4. Conditional expectations and Conditional expectation: existence and Properties of the conditional The conditional expectation as an orthogonal Regular conditional Probability distributions171 Chapter 5.
2 Discrete time martingales and stopping Definitions and closure Martingale representations and The convergence of The optional stopping Reversed MGs, likelihood ratios and branching processes212 Chapter 6. Markov Canonical construction and the strong Markov Markov chains with countable state General state space: Doeblin and Harris chains257 Chapter 7. Ergodic measure preserving and ergodic Birkhoff s ergodic Stationarity and The subadditive ergodic theorem(Draft: work in progres)281 Chapter 8. Continuous, Gaussian and stationary Definition, canonical construction and Continuous and separable Gaussian and stationary processes300 Chapter 9. Continuous time martingales and Markov Continuous time filtrations and stopping Continuous time Markov and Strong Markov processes334 Chapter 10.
3 The Brownian Brownian transformations, hitting times and Weak convergence and invariance Brownian path: regularity, local maxima and level sets385 Bibliography393 Index395310b: Homework Solutions 2021401 PrefaceThese are the lecture notes for a year long, PhD level course in Probability Theorythat I taught at Stanford University in 2004, 2006 and 2009. The goal of thiscourse is to prepare incoming PhD students in Stanford s mathematics and statisticsdepartments to do research in Probability theory . More broadly, the goal of the textis to help the reader master the mathematical foundations of Probability theoryand the techniques most commonly used in proving theorems in this area. This isthen applied to the rigorous study of the most fundamental classes of this goal, we introduce in Chapter 1 the relevant elements from measureand integration theory , namely, the Probability space and the -algebras of eventsin it, random variables viewed as measurable functions, their expectation as thecorresponding Lebesgue integral, and the important concept of these elements, we study in Chapter 2 the various notions ofconvergenceof random variables and derive the weak and strong laws of large 3 is devoted to the theory of weak convergence, the related conceptsof distribution and characteristic functions and two important special cases.
4 TheCentral Limit Theorem (in shortclt) and the Poisson upon the framework of Chapter 1, we devote Chapter 4 tothe definition,existence and properties of the conditional expectation and the associated regularconditional Probability 5 deals with filtrations, the mathematical notion of information progres-sion in time, and with the corresponding stopping times. Results about the latterare obtained as a by product of the study of a collection of stochastic processescalled martingales. Martingale representations are explored, as well as maximalinequalities, convergence theorems and various applications thereof. Aiming for aclearer and easier presentation, we focus here on the discrete time settings deferringthe continuous time counterpart to Chapter 6 provides a brief introduction to the theory of Markov chains, a vastsubject at the core of Probability theory , to which many text books are illustrate some of the interesting mathematical properties of such processes byexamining a few special cases of Chapter 7 we provide a brief introduction to Ergodic theory , limiting ourattention to its application for discrete time stochastic processes.
5 We define thenotion of stationary and ergodic processes, derive the classical theorems of Birkhoffand Kingman, and highlight few of the many useful applications that this 8 sets the framework for studying right-continuous stochastic processesindexed by a continuous time parameter, introduces the family of Gaussian pro-cesses and rigorously constructs the Brownian motion as a Gaussian process ofcontinuous sample path and zero-mean, stationary independent 9 expands our earlier treatment of martingales and strong Markov pro-cesses to the continuous time setting, emphasizing the role of right-continuous fil-tration. The mathematical structure of such processes is then illustrated both inthe context of Brownian motion and that of Markov jump on this, in Chapter 10 we re-construct the Brownian motion via theinvariance principle as the limit of certain rescaled random walks.
6 We further delveinto the rich properties of its sample path and the many applications of Brownianmotion to thecltand the Law of the Iterated Logarithm (in short,lil).The intended audience for this course should have prior exposure to stochasticprocesses, at an informal level. While students are assumed to have taken a realanalysis class dealing with Riemann integration, and mastered well thismaterial,prior knowledge of measure theory is not is quite clear that these notes are much influenced by the text books [Bil95,Dur10, Wil91, KaS97] I have been thank my students out of whose work this text materialized and myteaching as-sistants Su Chen, Kshitij Khare, Guoqiang Hu, Julia Salzman, Kevin Sun and HuaZhou for their help in the assembly of the notes of more than eighty students intoa coherent document.
7 I am also much indebted to Kevin Ross, Andrea Montanariand Oana Mocioalca for their feedback on earlier drafts of these notes, to KevinRoss for providing all the figures in this text, and to Andrea Montanari, DavidSiegmund and Tze Lai for contributing some of the exercises in these DemboStanford, CaliforniaApril 2010 CHAPTER 1 Probability , measure and integrationThis chapter is devoted to the mathematical foundations of Probability introduces the basic measure theory framework, namely, the probabilityspace and the -algebras of events in it. The next building blocks are randomvariables, introduced in Section as measurable functions 7 X( ) and allows us to define in Section the important concept of expectation as thecorresponding Lebesgue integral, extending the horizon of our discussion beyondthe special functions and variables with density to which elementaryprobabilitytheory is limited.
8 Section concludes the chapter by considering independence,the most fundamental aspect that differentiates Probability from(general) measuretheory, and the associated product Probability spaces, measures and -algebrasWe shall define here the Probability space ( ,F,P) using the terminology of mea-sure space is a set of all possible outcomes of some random exper-iment. Probabilities are assigned byA7 P(A) toAin a subsetFof all possiblesets of outcomes. Theevent spaceFrepresents both the amount of informationavailable as a result of the experiment conducted and the collection of all subsetsof possible interest to us, where we denote elements ofFasevents. A pleasantmathematical framework results by imposing onFthe structural conditions of a -algebra, as done in Subsection The most common and useful choices forthis -algebra are then explored in Subsection Subsection provides fun-damental supplements from measure theory , namely Dynkin s and Carath eodory stheorems and their application to the construction of Lebesgue The Probability space ( ,F, P).
9 We use 2 to denote the set of allpossible subsets of . The event space is thus a subsetFof 2 , consisting of allallowed events, that is, those subsets of to which we shall assign next define the structural conditions imposed say thatF 2 is a -algebra(or a -field), if(a) F,(b) IfA FthenAc Fas well (whereAc= \A).(c) IfAi Ffori= 1,2,3,..then alsoSiAi s law, we know that (SiAci)c=TiAi. Thus thefollowing is equivalent to property (c) of Definition :(c ) IfAi Ffori= 1,2,3,..then alsoTiAi Probability , measure AND pair( ,F)withFa -algebra of subsets of is called ameasurable space. Given a measurable space( ,F), ameasure is anycountablyadditivenon-negativeset functionon this space. That is, :F [0, ], havingthe properties:(a) (A) ( ) = 0for allA F.
10 (b) (SnAn) =Pn (An)for any countable collection of disjoint setsAn in addition ( ) = 1, we call the measure aprobability measure , andoften label it byP(it is also easy to see that thenP(A) 1for allA F). (b) of Definition is relaxed to involve only finite collectionsof disjoint setsAn, we say that is afinitely additivenon-negative measure theory we sometimes considersigned measures, whereby is no longernon-negative, hence its range is [ , ], and say that such measure isfinitewhenits range isR( no set inFis assigned an infinite measure ). spaceis a triplet( ,F, ), with a measure on themeasurable space( ,F). A measure space ( ,F,P) withPa Probability measureis called aprobability next exercise collects some of the fundamental properties shared by all prob-ability ( ,F,P)be a Probability space andA,B,Aievents the following properties of every Probability measure .
