Transcription of Stochastic Calculus, Filtering, and Stochastic Control
1 StochasticCalculus, Filtering, andStochast icControlLecture Notes(Thisversion:May29,2007)RamonvanHan delSpring2007 PrefaceTheselecturenoteswerewrittenforth ecourseACM217:AdvancedTopicsin Stochas-ticAnalysisat Caltech;thisyear(2007), anintroductorycourseonthesubject,andasth ereareonlysomany weeksina term, suf cientdodevelopa largeclassofinterestingmodels,andto developsomestochasticcontroland ,andmany otherfascinating(anduseful)topics,arelef tfora , thestochasticcontrolportionofthesenotesc oncentratesonveri- cationtheorems, hope,however, thattheinterestedreaderwillbeencouragedt oprobea littledeeperandultimatelytomove have noillusionsaboutthestateofthesenotes they werewrittenratherquickly,sometimesatther ateofa chaptera have nodoubtthatmany errorsremaininthetext;at theveryleastmany oftheproofsareextremelycompact,andshould bemadea littleclearerasis be ttingofa pedagogical(?)
2 I have anotheropportunitytoteachsucha course,I willgoover thenotesagainindetailandattemptthenecess arymodi ,however, youhave any commentsat allaboutthesenotes questions,suggestions,omis-sions,general comments,andparticularlymistakes Iwouldlove assumethatthereaderhashada basiccourseinprobabil-itytheoryat thelevel of,say, GrimmettandStirzaker [GS01] orhigher(ACM116/216shouldbesuf cient).Someelementarybackgroundinanalysi sis bitofanexperiment,butappearstohave , .. ,independence,andabsolutecontinuity.. technicaltool:Dynkin's -systemlemma..442 Conditioning,Martingales, :a trialrun.. :a multiscaleconstruction..854 TheIt o wrongwiththeStieltjesintegral?.. o integral.. o calculus.. 's theorem.. :existenceanduniqueness.
3 PropertyandKolmogorov's equations.. therelifebeyondtheLipschitzcondition?.. cation: nitetimehorizon.. cation:inde nitetimehorizon.. cation:in nitetimehorizon.. chainapproximation.. lteringforstochasticdifferentialequation s.. lter.. lter.. messageovera noisychannel.. :themodi cationproblem.. ,stochasticcalculusprovidesa mathematicalfoundationforthetreatmentofe quationsthatinvolve ,bothmathematicalandpractical, , ,tracking,and nanceBrownianmotionandtheWienerprocessIn 1827,the(thenalready)famousScottishbotan istRobertBrownobserveda rathercuriousphenomenon[Bro28]. Brownwasinterestedinthetiny particlesfoundinsidegrainsofpollen, ,it appearedthattheparticleswerecon-stantlyj itteringaroundin the rstBrownthoughtthattheparticleswerealive ,buthewasabletoruleoutthishypothesisafte rheobservedthesamephenomenonwhenusinggla sspowder, anda largenumberofotherinorganicsubstances, 's observationwasnotprovideduntilthepublica tionofEinstein's famous1905paper[Ein05].
4 Einstein's argumentreliesonthefactthatthe uid,inwhichthepollenparti-clesaresuspend ed,consistsofa giganticnumberofwatermolecules(thoughthi sisnow undisputed,theatomichypothesiswashighlyc ontroversialatthetime).Asthe uidis at a nitetemperature,kinetictheorysuggeststha tthevelocityofeverywatermoleculeis randomlydistributedwithzeromeanvalue(the lattermustbethecase,asthetotal uidhasnonetvelocity)andis weplacea pollenparticleinthe uid,thenineverytimeinter-valtheparticlew illbebombardedbya largenumberofwatermolecules,givingita shouldwegoaboutmodellingthisphenomenon?T hefollowingprocedure,whichis a somewhatmodernizedversionofEinstein's argument,is bombardedbyNwatermoleculesperunittime,an dthateverywatermoleculecontributesaninde pen-dent,identicallydistributed( )randomdisplacement ntotheparticle(where (N)fortheBrownianmotionmodelinthetext,wi th(fromlefttoright)N= 20;50; narechosentoberandomvariableswhichtake thevalues N 1= ).
5 Thenat timet, thepositionxt(N)ofthepollenparticleis givenbyxt(N) =x0+bN tcXn=1 n:We wanttoconsiderthelimitwherethenumberofbo mbardmentsNis verylarge,butwhereeveryindividualwatermo leculeonlycontributesa tiny displacementtothepollenparticle thisis a reasonableassumption,asthepollenparticle ,whilebeingsmall,is extremelylargecomparedtoa beconcrete,letusde nea constant byvar( n) = N 1. Notethat is preciselythemean-squaredisplacementofthe pollenparticleperunittime:E(x1(N) x0)2= var NXn=1 n!=Nvar( n) = :Thephysicalregimein whichweareinterestednow correspondsto thelimitN!1, ,wherethenumberofcollisionsNis largebutthemean-squaredisplacementperuni ttime remains (N) =x0+p tPbN tcn=1 npNt;where n= npN= ,weseethatthelimitingbehaviorofxt(N)asN!
6 1is describedbythecentrallimittheorem:we ndthatthelaw ofxt(N)convergestoa Gaussiandistributionwithzeromeanandvaria nce t. Thisis indeedtheresultofEinstein's ! 1is knownasBrownianmo-tion. Youcangetsomeideaofwhatxt(N)lookslike forincreasinglylargeNbyhavinga lookat Butnowwecometoour rstsigni cantmathematicalproblem:doesthelimitofth estochasticprocesst7!xt(N)asN! 1evenexistina suitablesense?Thisis notatallobvious(wehave onlyshownconvergenceinIntroduction3distr ibutionfor xedtimet), noris wecanmake nosenseofthislimit,therewouldbenomathema ticalmodelofBrownianmotion(aswehave de nedit);andinthiscase, mathematicalsenseofBrownianmotion(chapte r3), whichwas rstdoneinthefundamentalworkofNorbertWien er[Wie23]. Thelimitingstochasticprocessxt(with = 1) is knownastheWienerprocess, andplaysa diffusingparticleUsingonlythenotionofa Wienerprocess, ,like RobertBrown, ,wewouldlike tozoominononeoftheparticles ,wewouldlike toincreasethemagni cationofthemicroscopeuntilonepollenparti cle llsa largepartofthe eldofview.
7 Whenwedothis,however, theBrownianmotionbecomesa bitofa nuisance;therandommotionofthepollenparti clecausesit torapidlyleave our eldofview. If wewanttokeeplookingat thepollenparticlefora reasonableamountoftime,wehave to dealwiththisproblem,weattachanelectricmo tortothemicroscopeslidewhichallowsustomo ve ; thenwecanwritedztdt= ut;whereutis thevoltageappliedto themotorand >0is a totheslideis modelledbya Wienerprocessxt, sothatthepositionoftheparticlerelative tothemicroscopefocusis givenbyxt+zt. Wewouldlike tocontroltheslidepositiontokeepthepartic leinfocus, ,it is ourgoaltochooseutinorderthatxt+ formalizethisproblem,wecouldintroducethe followingcostfunctional:JT[u] =pE"1 TZT0(xt+zt)2dt#+qE"1 TZT0u2tdt#;wherepandqaresomepositive rstterminthisexpressionisthetime-average (onsometimeinterval[0;T]) meansquaredistanceoftheparticlefromthefo cusofthemicroscope:clearlywewouldlike ,ontheotherhand,is theaveragepowerinthecontrolsignal,whichs houldalsonotbetoolargeinany realisticapplication(ourelectricmotorwil lonlytake somuch).
8 Thegoaloftheoptimalcontrolproblemisto ndthefeedbackstrategyutwhichminimizesthe costJT[u]. Many variationsonthisproblemarepossible;forex ample,if wearenotinterestedina particulartimehorizon[0;T], wecouldtrytominimizeJ1[u] =plimsupT!1E"1 TZT0(xt+zt)2dt#+qlimsupT!1E"1 TZT0u2tdt#:Introduction4 Thetradeoff betweenthecon ictinggoalsofminimizingthedistanceofthep articletothefocusofthemicroscopeandminim izingthefeedbackpowercanbeselectedbymodi fyingtheconstantsp;q. Theoptimalcontroltheoryfurtherallowsusto studythistradeoff explicitly:forexample,wecancalculatetheq uantityC(U) = inf(limsupT!1E"1 TZT0(xt+zt)2dt#: limsupT!1E"1 TZT0u2tdt# U); ,C(U)isthesmallesttime-averagetrackinger rorthatisachievableusingcon-trolswhoseti me-averagepoweris atmostU.
9 Thisgivesa ,ina muchmoregeneralcontext,is :how toinvestyourmoneyThoughthetheoreticalide asbehindBrownianmotionareoftenattributed toEinstein,thesamemodelwasdevelopedsever alyearsearlierina completelydifferentcon-textbytheFrenchma thematicianLouisBachelier[Bac00].1 Bachelierwasinterestedinspeculationonren tes(Frenchgovernmentbonds),andintroduced theBrownianmotiontomodelthe 's workformsthefoun-dationformuchofthemoder ntheoryofmathematical nance,thoughhisworkwasvirtuallyunknownto economistsformorethanhalfa century. Thesedaysmathe-matical nanceis animportantapplicationareaforstochastica nalysisandstochasticcontrol,andprovidesa nancetheirop-erations;themoney madefromthesaleofstockcanthenbeusedbythe company to nanceproduction,specialprojects, ,a certainportionofthepro tmadebythecompany is periodicallypaidouttotheshareholders(peo plewhoownstockinthecompany).
10 Thecompany is doingwell( ,if salesaresoaring),thenowningstockinthecom pany is likelytobepro onlythebeginningofthestory, however. Any individualwhoownsstockina company candecidetosellhisstockona ,thegoingratefora particularstockdependsonhow wellthecompany is doing(orisexpectedtodointhefuture).Whent hecompany is doingwell,many peoplewouldlike toownstock(afterall,thereis a prospectoflargedividends) is notdoingwell,however, it is likelythatmoreshareholdersarewillingtose llthanthereis demandforthestock,sothatthemarket priceofthestockis these market forces ,thestockpricestendto uctuaterandomlyinthecourseoftime;see,for example, Evenif weignoredividends(whichwewilldotosimplif ymatters),wecanstilltrytomake money onthestockmarketbybuyingstockwhenthepric eis low andsellingwhenthepriceis ,how shouldweinvestourmoney in thestockmarket to maximizeourpro t?
