Tutorial on Stochastic Di erential Equations - johnboccio.com

1 Please cite as J. R. Movellan (2011) Tutorial on Stochastic Differential Equations ,MPLab Tutorials Version on Stochastic DifferentialEquationsJavier R. MovellanCopyrightc 2003, 2004, 2005, 2006, Javier R. MovellanThis document is being reorganized. Expect redundancy, inconsistencies, disorga-nized presentation ..1 MotivationThere is a wide range of interesting processes in robotics, control, economics, thatcan be described as a differential Equations with non-deterministic dynamics. Sup-pose the original processes is described by the following differential equationdXtdt=a(Xt)(1)with initial conditionX0, which could be random. We wish to construct a math-ematical model of how the may behave in the presence of noise.

2 We wish for thisnoise source to be stationary and independent of the current state of the also want for the resulting paths to be it turns out building such a model is tricky. An elegant mathematical solutionto this problem may be found by considering a discrete time versions of the processand then taking limits in some meaningful way. Let ={0 =t0 t1 tn=t}be a partition of the interval [0,t]. Let tk=tk+1 tk. For each partition wecan construct a continuous time processX defined as followsX t0=X0(2)X tk+1=X tk+a(X tk) tk+c(X tk)(Ntk+1 Ntk)(3)whereNis a noise process whose properties remain to be determined andbis afunction that allows us to have the amount of noise be a function of time and of thestate.

3 To make the process be continuous in time, we make it piecewise constantbetween the intervals defined by the partition, t=X tkfort [tk,tk+1)(4)We want for the noiseNtto be continuous and for the incrementsNtk+1 Ntktohave zero mean, and to be independently and identically distributed. It turns outthat the only noise source that satisfies these requirements is Brownian we getX t=X0+n 1 k=0a(X tk) tk+n 1 k=0c(X tk) Bk(5)where tk=tk+1 tk, and Bk=Btk+1 BtkwhereBis Brownian Motion. Let = max{ tk}be the norm of the partition . It can be shown that as 0theX processes converge in probability to a Stochastic processX. It follows thatlim 0n 1 k=0a(X tk) k= t0a(Xs)ds(6)and thatn 1 k=0c(X tk) Bk(7)converges to a processItIt= lim 0n 1 k=0c(X tk) Bk(8)NoteItlooks like an integral where the integrand is a random variablec(Xs) andthe integrator Bkis also a random variable.]

4 As we will see later,Itturns out tobe an Ito Stochastic Integral. We can now express the limit processXas a processsatisfying the following equationXt=X0+ t0a(Xs)ds+It(9)Sketch of Proof of Convergence:Construct a sequence of partitions 1, 2, eachone being a refinement of the previous one. Show that the correspondingX itform a Cauchy sequence inL2and therefore converge to a limit. Call that order to get a better understanding of the limit processXthere are two thingswe need to do: (1) To study the properties of Brownian motion and (2) to studythe properties of the Ito Stochastic Standard Brownian motionBy Brownian motion we refer to a mathematical model of the random movementof particles suspended in a fluid.

5 This type of motion was named after RobertBrown that observed it in pollens of grains in water. The processes was describedmathematically by Norbert Wiener, and is is thus also called a Wiener a standard Brownian motion (or Wiener Process) is defined by thefollowing properties:1. The process starts at zero with probability 1, ,P(B0= 0) = 12. The probability that a randomly generated Brownian path be continuousis The path increments are independent Gaussian, zero mean, with varianceequal to the temporal extension of the increment. Specifically for 0 s1 t1 s2, t2Bt1 Bs1 N(0,s1 t1)(10)Bt2 Bs2 N(0,s2 t2)(11)andBt2 Bs2is independent ofBt1 showed that such a process exists, , there is a Stochastic process that doesnot violate the axioms of probability theory and that satisfies the 3 Properties of Brownian StatisticsFrom the properties of Gaussian random variables,E(Bt Bs) = 0(12)Var(Bt Bs) =E[(Bt Bs)2] =t s(13)E((Bt Bs)4] = 3(t s)(14)Var[(Bt Bs)2] =E[(Bt Bs)4] E[(Bt Bs)2]2= 2(t s)2(15)Cov(Bs,Bt) =s,fort > s(16)Corr(Bs,Bt) = st,fort > s.)

6 (17)Proof:For the variance of (Bt Bs)2we used the that for a standard randomvariableZE(Z4) = 3(18)NoteVar(BT) = Var(BT B0) =T(19)sinceP(B0= 0) and for all t 0 Var(Bt+ t Bt) = t(20)Moreover,Cov(Bs,Bt) = Cov(Bs,Bs+ (Bt Bs)) = Cov(Bs,Bs) + Cov(Bs,(Bt Bs))= Var(Bs) =s(21)sinceBsandBt Bsare Distributional PropertiesLetBrepresent a standard Brownian motion (SBM) process. Self-similarity:For anyc6= 0,Xt=1 cBctis can use this property to simulate SBM in any given interval [0,T] if weknow how to simulate in the interval [0,1]:IfBis SBM in [0,1],c=1 TthenXt= T B1 Ttis SBM in [0,T]. Time Inversion:Xt=tB1tis SBM Time Reversal:Xt=BT BT tis SBM in the interval [0,T] Symmetry:Xt= Btis Pathwise Properties Brownian motion sample paths are non-differentiable with probability 1 This is the basic why we need to develop a generalization of ordinary calculus tohandle Stochastic differential Equations .

7 If we were to define such Equations simplyasdXtdt=a(Xt) +c(Xt)dBtdt(22)we would have the obvious problem that the derivative of Brownian motion doesnot :LetXbe a real valued Stochastic process. For a fixedtlet ={0 =t0 t1, tn=t}be a partition of the interval [0,t]. Let be the norm of thepartition. The quadratic variation ofXattis a random variable represented as< X,X >2tand defined as follows< X,X >2t= lim 0n k=1|Xtk+1 Xtk|2(23)We will show that the quadratic variation of SBM is larger than zero with probabilityone, and therefore the quadratic paths are not differentiable with probability a Standard Brownian Motion. For a partition ={0 =t0 t1, tn=t}letB kbe defined as followsB k=Btk(24)LetS =n k=1( B k)2(25)NoteE(S ) =n 1 k=0tk+1 tk=t(26)and0 Var(S ) =n 1 k=0 Var[( B k)2]= 2n 1 k=0(tk+1 tk)2 2 n 1 k=0(tk+1 tk) = 2 t(27)Thuslim 0 Var(S ) = lim 0E (n 1 k=0( B k)2 t)2 = 0(28)This shows mean square convergence, which implies convergence in probability, ofS tot.

8 (I think) almost sure convergence can also be : If we were to define the Stochastic integral t0(dBs)2as t0(dBs)2= lim 0S (29)Then s0(dBs)2= t0ds=t(30) If a pathXt( ) were differentiable almost everywhere in the interval [0,T]then< X,X >2t( )) lim t 0n 1 k=0( tX tk( ))2(31)= ( maxt [0,T]X t( )2) limn 2t(32)= ( maxt [0,T]X t( )2) limn (n)(T/n)2= 0(33)whereX =dX/dt. Since Brownian paths have non-zero quadratic varia-tion with probability one, they are also non-differentiable with Simulating Brownian MotionLet ={0 =t0 t1 tn=t}be a partition of the interval [0,t]. Let{Z1, ,Zn}; be Gaussian random variablesE(Zi) = 0;Var(Zi) = 1. Letthe Stochastic processB as follows,B t0= 0(34)B t1=B t0+ t1 t0Z1(35).

9 (36)B tk=B tk 1+ tk tk 1Zk(37)Moreover,B t=B tk 1fort [tk 1,tk)(38)For each partition this defines a continuous time process. It can be shown that as 0 the processB converges in distribution to Standard Brownian ExerciseSimulate Brownian motion and verify numerically the following propertiesE(Bt) = 0(39)Var(Bt) =t(40) t0dB2s= s0ds=t(41)3 The Ito Stochastic IntegralWe want to give meaning to the expression t0 YsdBs(42)whereBis standard Brownian Motion andYis a process that does not anticipatethe future of Brownian motion. For example,Yt=Bt+2would not be a validintegrand. A random processYis simply a set of functionsf(t, ) from an outcomespace to the real numbers, for each Yt( ) =f(t, )(43)We will first study the case in whichfis piece-wise constant.]

10 In such case there isa partition ={0 =t0 t1 tn=t}of the interval [0,.t] such thatfn(t, ) =n 1 k=0Ck( ) k(t)(44)where k(t) ={1 ift [tk,tk+1)0 else(45)whereCkis a non-anticipatory random variable, , a function ofX0and theBrownian noise up to timetk. For such a piece-wise constant processYt( ) =fn(t, ) we define the Stochastic integral as follows. For each outcome t0Ys( )dBs( ) =n 1 k=0Ck( )(Btk+1( ) Btk( ))(46)More succinctly t0 YsdBs=n 1 k=0Ck(Btk+1 Btk)(47)This leads us to the more general definition of the Ito integralDefinition of the Ito IntegralLetf(t, ) be a non-anticipatory function from anoutcome space to the real numbers. Let{f1,f2, }be a sequence of elementarynon-anticipatory functions such thatlimn E[ t0(f(s, ) fn(s, ))2ds] = 0(48)Let the random processYbe defined as follows:Yt( ) =f(t, ) Then the Itointegral t0 YsdBs(49)is a random variable defined as follows.]}