A TUTORIAL INTRODUCTION TO STOCHASTIC …

1 A TUTORIAL INTRODUCTION TO STOCHASTICANALYSIS AND ITS APPLICATIONSbyIOANNIS KARATZASD epartment of StatisticsColumbia UniversityNew York, 10027 September 1988 SynopsisWe present in these lectures, in an informal manner, the very basic ideas and results ofstochastic calculus, including its chain rule, the fundamental theorems on the represen-tation of martingales as STOCHASTIC integrals and on the equivalent change of probabilitymeasure, as well as elements of STOCHASTIC differential equations. These results suffice fora rigorous treatment of important applications, such as filtering theory, STOCHASTIC con-trol, and the modern theory of financial economics.

2 We outline recent developments inthese fields, with proofs of the major results whenever possible, and send the reader to theliterature for further familiarity with probability theory and STOCHASTIC processes , including a goodunderstanding of conditional distributions and expectations, will be assumed. Previousexposure to the fields of application will be desirable, but not necessary. Lecture notes prepared during the period 25 July - 15 September 1988, while the authorwas with the Office for Research & Development of the Hellenic Navy ( ETEN), at thesuggestion of its former Director, Capt.

3 I. Martinos. The author expresses his appreciationto the leadership of the Office, in particular Capts. I. Martinos and A. Nanos, Cmdr. , and Dr. B. Sylaidis, for their interest and Brownian Motion (Wiener process)..63. STOCHASTIC The Chain Rule of the new The Fundamental Dynamical Systems driven by White Noise Filtering Robust STOCHASTIC AND SUMMARYThe purpose of these notes is to introduce the reader to the fundamental ideas and resultsofStochastic Analysisup to the point that he can acquire a working knowledge of thisbeautiful subject, sufficient for the understanding and appreciation of its r ole in importantapplications.

4 Such applications abound, so we have confined ourselves to only two of them,namelyfiltering theoryandstochastic control; this latter topic will also serve us as a vehiclefor introducing important recent advances in the field of financial economics, which havebeen made possible thanks to the methodologies of STOCHASTIC have adopted an informal style of presentation, focusing on basic results and onthe ideas that motivate them rather than on their rigorous mathematical justification, andproviding proofs only when it is possible to do so with a minimum of technical the reader who wishes to undertake an in-depth study of the subject.

5 There are nowseveral monographs and textbooks available, such as Liptser & Shiryaev (1977), Ikeda &Watanabe (1981), Elliott (1982) and Karatzas & Shreve (1987).The notes begin with a review of the basic notions of Markov processes and martin-gales (section 1) and with an outline of the elementary properties of their most famousprototype, the Wiener-L evy or Brownian Motion process (section 2). We then sketchthe construction and the properties of the integral with respect to this process (section3), and develop the chain rule of the resulting STOCHASTIC calculus (section 4).

6 Section5 presents the fundamental representation properties for continuous martingales in termsof Brownian motion (via time-change or integration), as well as the celebrated result ofGirsanov on the equivalent change of probability measure. Finally, we offer in section 6 anelementary study of dynamical systems excited by white noise 7 applies the results of this theory to the study of the filtering problem. Thefundamental equations of Kushner and Zakai for the conditional distribution are obtained,and the celebrated Kalman-Bucy filter is derived as a special (linear) case.

7 We also outlinethe derivation of the genuinely nonlinear Bene s (1981) filter, which is nevertheless explicitlyimplementable in terms of a finite number of sufficient statistics. A reduction of the filteringequations to a particularly simple form is presented in section 8, under the rubric of robustfiltering , and its significance is demonstrated on INTRODUCTION to STOCHASTIC control theory is offered in section 9; we present theprinciple ofDynamic Programmingthat characterizes the value function of this problem,and derive from it the associated Hamilton-Jacobi-Bellman equation.

8 The notion of weaksolutions (in the viscosity sense of Lions) of this equation is expounded upon. Inaddition, several examples are presented, including the so-called linear regulator and theportfolio/consumption problem from financial GENERALITIESA STOCHASTIC process is a family of random variablesX={Xt; 0 t < }, , ofmeasurable functionsXt( ) : R, defined on a probability space ( ,F,P). For every , the functiont7 Xt( ) is called thesample path(or trajectory) of the Example:LetT1,T2, (independent, identically distributed) randomvariables with exponential distributionP(Ti dt) = e tdt, fort >0, and defineS0( ) = 0, Sn( ) = nj=1Tj( ) forn interpretation here is that theTj s represent the interarrival times, and that theSn srepresent the arrival times, of customers in a certain facility.

9 The STOCHASTIC processNt( ) = #{n 1 :Sn( ) t},0 t < counts, for every 0 t < , the number of arrivals up to that time and is called aPoissonprocesswith intensity >0. Every sample patht7 Nt( ) is a staircase function (piecewise constant, right-continuous, with jumps of size +1 at the arrival times), andwe have the following properties:(i) for every 0 =t0< t1< t2< < tm< t < < , the incrementsNt1,Nt2 Nt1, , Nt Ntm, N Ntare independent;(ii) the distribution of the incrementN Ntis Poisson with parameter ( t), ,P[N Nt=k] =e ( t)( ( t))kk!

10 , k= 0,1,2, .It follows from the first of these properties thatP[N =k|Nt1,Nt2,..,Nt] =P[N =k|Nt1,Nt2 Nt1, ,Nt Ntm,Nt] =P[N =k|Nt],and more generally, withFNt= (Ns; 0 s t):( )P[N =k|FNt] =P[N =k|Ns; 0 s t] =P[N =k|Nt].In other words, given the past {Ns: 0 s < t}and the present {Nt}, the future {N }depends only on the present. This is theMarkov propertyof the Poisson Remark on Notation:For every STOCHASTIC processX, we denote by( )FXt= (Xs; 0 s t)the record (history, observations, sample path) of the process up to timet.