A TUTORIAL INTRODUCTION TO STOCHASTIC ANALYSIS …

1 A TUTORIAL INTRODUCTION TO STOCHASTICANALYSIS AND ITS APPLICATIONSbyIOANNIS KARATZASD epartment of StatisticsColumbia UniversityNew York, 10027 September 1988 SynopsisWe present in these lectures, in an informal manner, the very basic ideas and results ofstochastic calculus, including its chain rule, the fundamental theorems on the represen-tation of martingales as STOCHASTIC integrals and on the equivalent change of probabilitymeasure, as well as elements of STOCHASTIC differential equations. These results suffice fora rigorous treatment of important applications, such as filtering theory, STOCHASTIC con-trol, and the modern theory of financial economics. We outline recent developments inthese fields, with proofs of the major results whenever possible, and send the reader to theliterature for further familiarity with probability theory and STOCHASTIC processes, including a goodunderstanding of conditional distributions and expectations, will be assumed.

2 Previousexposure to the fields of application will be desirable, but not necessary. Lecture notes prepared during the period 25 July - 15 September 1988, while the authorwas with the Office for Research & Development of the Hellenic Navy ( ETEN), at thesuggestion of its former Director, Capt. I. Martinos. The author expresses his appreciationto the leadership of the Office, in particular Capts. I. Martinos and A. Nanos, Cmdr. , and Dr. B. Sylaidis, for their interest and Brownian Motion (Wiener process)..63. STOCHASTIC The Chain Rule of the new The Fundamental Dynamical Systems driven by White Noise Filtering Robust STOCHASTIC AND SUMMARYThe purpose of these notes is to introduce the reader to the fundamental ideas and resultsofStochastic Analysisup to the point that he can acquire a working knowledge of thisbeautiful subject, sufficient for the understanding and appreciation of its r ole in importantapplications.

3 Such applications abound, so we have confined ourselves to only two of them,namelyfiltering theoryandstochastic control; this latter topic will also serve us as a vehiclefor introducing important recent advances in the field of financial economics, which havebeen made possible thanks to the methodologies of STOCHASTIC have adopted an informal style of presentation, focusing on basic results and onthe ideas that motivate them rather than on their rigorous mathematical justification, andproviding proofs only when it is possible to do so with a minimum of technical the reader who wishes to undertake an in-depth study of the subject, there are nowseveral monographs and textbooks available, such as Liptser & Shiryaev (1977), Ikeda &Watanabe (1981), Elliott (1982) and Karatzas & Shreve (1987).The notes begin with a review of the basic notions of Markov processes and martin-gales (section 1) and with an outline of the elementary properties of their most famousprototype, the Wiener-L evy or Brownian Motion process (section 2).

4 We then sketchthe construction and the properties of the integral with respect to this process (section3), and develop the chain rule of the resulting STOCHASTIC calculus (section 4). Section5 presents the fundamental representation properties for continuous martingales in termsof Brownian motion (via time-change or integration), as well as the celebrated result ofGirsanov on the equivalent change of probability measure. Finally, we offer in section 6 anelementary study of dynamical systems excited by white noise 7 applies the results of this theory to the study of the filtering problem. Thefundamental equations of Kushner and Zakai for the conditional distribution are obtained,and the celebrated Kalman-Bucy filter is derived as a special (linear) case. We also outlinethe derivation of the genuinely nonlinear Bene s (1981) filter, which is nevertheless explicitlyimplementable in terms of a finite number of sufficient statistics.

5 A reduction of the filteringequations to a particularly simple form is presented in section 8, under the rubric of robustfiltering , and its significance is demonstrated on INTRODUCTION to STOCHASTIC control theory is offered in section 9; we present theprinciple ofDynamic Programmingthat characterizes the value function of this problem,and derive from it the associated Hamilton-Jacobi-Bellman equation. The notion of weaksolutions (in the viscosity sense of Lions) of this equation is expounded upon. Inaddition, several examples are presented, including the so-called linear regulator and theportfolio/consumption problem from financial GENERALITIESA STOCHASTIC process is a family of random variablesX={Xt; 0 t < }, , ofmeasurable functionsXt( ) : R, defined on a probability space ( ,F,P).

6 For every , the functiont7 Xt( ) is called thesample path(or trajectory) of the Example:LetT1,T2, (independent, identically distributed) randomvariables with exponential distributionP(Ti dt) = e tdt, fort >0, and defineS0( ) = 0, Sn( ) = nj=1Tj( ) forn interpretation here is that theTj s represent the interarrival times, and that theSn srepresent the arrival times, of customers in a certain facility. The STOCHASTIC processNt( ) = #{n 1 :Sn( ) t},0 t < counts, for every 0 t < , the number of arrivals up to that time and is called aPoissonprocesswith intensity >0. Every sample patht7 Nt( ) is a staircase function (piecewise constant, right-continuous, with jumps of size +1 at the arrival times), andwe have the following properties:(i) for every 0 =t0< t1< t2< < tm< t < < , the incrementsNt1,Nt2 Nt1, , Nt Ntm, N Ntare independent;(ii) the distribution of the incrementN Ntis Poisson with parameter ( t), ,P[N Nt=k] =e ( t)( ( t))kk!

7 , k= 0,1,2, .It follows from the first of these properties thatP[N =k|Nt1,Nt2,..,Nt] =P[N =k|Nt1,Nt2 Nt1, ,Nt Ntm,Nt] =P[N =k|Nt],and more generally, withFNt= (Ns; 0 s t):( )P[N =k|FNt] =P[N =k|Ns; 0 s t] =P[N =k|Nt].In other words, given the past {Ns: 0 s < t}and the present {Nt}, the future {N }depends only on the present. This is theMarkov propertyof the Poisson Remark on Notation:For every STOCHASTIC processX, we denote by( )FXt= (Xs; 0 s t)the record (history, observations, sample path) of the process up to timet. The resultingfamily{FXt; 0 t < }isincreasing:FXt FX fort < . This corresponds to theintuitive notion that( ){FXtrepresents the information about the processXthat has been revealed up to timet},4and obviously this information cannot decrease with shall write{Ft; 0 t < }, or simply{Ft}, whenever the specification of theprocess that generates the relevant information is not of any particular importance, andcall the resulting family afiltration.

8 Now ifFXt Ftholds for everyt 0, we say thatthe process X is adapted to the filtration{Ft}, and write{FXt} {Ft}. The Markov property:A STOCHASTIC processXis said to beMarkovian, ifP[X A|FXt] =P[X A|Xt]; A B(R),0< t < .Just like the Poisson process, every process with independent increments has this The Martingale property:A STOCHASTIC processXwithE|Xt|< is calledmartingale,ifE(Xt|Fs) =Xssubmartingale,ifE(Xt|Fs) Xssupermartingale,ifE(Xt|Fs) Xsholds ( ) for every 0< s < t < . Discussion:(i) The filtration{Ft}in can be the same as{FXt}, but it may also belarger. This point can be important ( in the representation Theorem ) or even crucial( in Filtering Theory; cf. section 7), and not just a mere technicality. We stress it, whennecessary, by saying that Xis an{Ft}- martingale , or that X={Xt,Ft; 0 t < }is a martingale.

9 (ii) In a certain sense, martingales are the constant functions of probability theory;submartingales are the increasing functions , and supermartingales are the decreasingfunctions . In particular, for a martingale (submartingale, supermartingale) the expecta-tiont7 EXtis a constant (resp. nondecreasing, nonincreasing) function; on the otherhand, a super(sub)martingale with constant expectation is necessarily a martingale. Withthis interpretation, ifXtstands for the fortune of a gambler at timet, then a marti-nale (submartingalge, supermartingale) corresponds to the notion of a fair (respectively:favorable, unfavorable) game.(iii) The study of processes of the martingale type is at the heart of STOCHASTIC ANALYSIS , andbecomes exceedingly important in applications. We shall try in this TUTORIAL to illustrateboth these The Compensated Poisson process:IfNis a Poisson process with intensity >0, it is checked easily that the compensated process Mt=Nt t,FNt,0 t < is a martingale.

10 In order to state correctly some of our later results, we shall need to localize themartingale Definition:A random variable : [0, ] is called astopping timeof the filtration{Ft}, if the event{ t}belongs toFt, for every 0 t < .In other words, the determination of whether has occurred by timet, can be madeby looking at the informationFtthat has been made available up to timetonly, withoutanticipation of the instance, ifXhas continuous paths andAis a closed set of the real line, the hitting time A= min{t 0 :Xt A}is a stopping Definition:An adapted processX={Xt,Ft; 0 t < }is called alocal martingale,if there exists an increasing sequence{ n} n=1of stopping times with limn n= suchthat the stopped process {Xt n,Ft; 0 t < }is a martingale, for everyn can be shown that every martingale is also a local martingale, and that there existlocal martingales which are not martingales.