SC505 STOCHASTIC PROCESSES Class Notes - mit.edu

1 SC505 STOCHASTICPROCESSESC lassNotesc Casta~non & Electricaland ComputerEngineeringBostonUniversityColle geof Engineering8 St. Mary'sStreetBoston,MA02215 Fall 20042 Contents1 Introductionto probability .. andIndependenceof Events.. RandomVariables.. RandomVariables.. RandomVariables.. ,Densities,andExpectations.. theCovarianceMatrix.. inequality.. Inequality .. 'sInequality .. Inequalities..372 Sequencesof .. of LargeNumbers.. Convergence.. theLaw of LargeNumbersandtheCentralLimitTheorem.. RandomVariables..523 .. StochasticProcesses.. of StochasticProcesses.. StochasticProcesses.. StochasticProcesses.. RandomProcesses.. :Phase-ShiftKeying.. Functionsof VectorProcesses.. of Wide-senseStationaryProcesses.

2 SpectralDensity of Wide-SenseStationaryProcesses..714 of StochasticProcesses.. erentiation.. erentiationof GaussianStochasticProcesses.. of StationaryRandomProcesses..865 .. Continuous-timeLinearSystems.. Discrete-timeLinearSystems.. MultivariableSystems..996 Samplingof .. 1057 ModelIdenti Models.. MovingAverage(ARMA)Models.. 1168 .. RiskApproach andtheLikelihood RatioTest.. OperatingCharacteristic.. theROC.. 1469 SeriesExpansionsandDetectionof .. StochasticProcesses.. KnownSignalsin Additive WhiteNoise.. UnknownSignalsin WhiteNoise.. KnownSignalsin ColoredNoise.. 157 CONTENTS510 Estimationof .. DecisionRule.. DecisionRulePerformance.. LeastSquareEstimation.

3 MaximumA Posteriori(MAP)Estimation.. LinearLeastSquare(LLSE)Estimation.. Estimation.. MAPestimation.. 18711 LLSEE stimationof .. :TheWiener-HopfEquation.. (WienerSmoothing).. 20912 Recursive .. Estimationof a RandomVector.. UpdateStep.. theWienerandKalmanFilter.. 22113 DiscreteStateMarkov ,DiscreteValuedMarkov PROCESSES .. ,DiscreteValuedMarkov PROCESSES .. PoissonProcesses.. 233A UsefulTransforms235B .. 242C Summaryof .. Transformation.. niteMatrices.. 254D Thenon-zeromeancase257 Listof Times k.. timesT(n) andinterarrival times k.. (PCP)N(t) andtherelationshipbetweenarrival timesT(n)andinterarrival times k.. a deterministicdecisionruleas a divisionof theobservationspaceinto disjointregions,illustratedhereforthecas eof two.

4 We obtainmoreindependent .. discretevaluedproblemof .. discretevaluedproblemof .. theperformanceof a .. theoverallROCobtainedfora .. :Illustrationof theexpectedcostof a decisionruleusinganarbitrary xedthresholdasa functionof thetruepriorprobabilityP 1. Themaximumcostof thisdecisionruleis at theleftendpoint. Thelower curve is thecorrespondingexpectedcostof theoptimalLRT. Right:Theexpectedcostof theminimaxdecisionruleas a functionof thetruepriorprobabilityP 1. by intersecting( ) fora .. ,decisionregionsandPFfortheproblemof .. thespaceof thelikelihoods .. thedecisionrulein .. thedecisionrulein thelikelihood .. theMLdecisionrulein .. decisionrulein.

5 Thecalculationof Pr (DecideH0jH1) in .. thecalculationof Pr (DecideH1jH1) in .. theMLEstimator:(a)pYjX(yjx) viewed as a functionofyfor xedvaluesofx, (b)pYjX(yjx) viewed as a functionofxfor xedy, (c)pYjX(yjx) viewed as a functionof bothxandy. For a givenobservationy0,bxM L(y) is themaximumwithrespecttoxforthegiveny=y0. . BasedonRelative Timesof .. noncausalWiener lterof .. spectraof .. to .. (t), thepole-zeroplot, .. (t), thepole-zeroplot, .. (t), thepole-zeroplot, .. (t), thepole-zeroplot, .. (t) forT >0 andT <0.. (s).. pole-zerosymmetrypropertiesofSY Y(s).. WienerFilterSolutions.. :Estimate.. :CovarianceandGain.. 229 Listof probability distributionfunctionandprobability density.

6 Randomvariables.(N/AunderthePDFcolumnind icatesthatthereis no simpli edform.).. MAPandMLEstimationfora .. thecausalWiener lterandtheKalman .. 23910 LISTOFTABLESC hapter1 Introductionto ProbabilityWhatis probability theory?It is anaxiomatictheorywhichdescribes andpredictstheoutcomesof inexact, theabove de probabilisticanalysisis todetermineorestimatetheprobabilitiestha tcertainknownevents occur,andthentousetheaxiomsofprobability theoryto combinethisinformationto derive probabilitiesof otherevents of interest,andtopredicttheoutcomesof example,considerany is theshu ingof a deck of cards,withtheoutcomebeingtheorderin which theunderlyingprobabilitieswouldbe thatallorderingsareequallylikely;theunde rlyingevents wouldthenbe assigneda of theevents,youmay wishto computetheprobability that,if youareplayingaloneagainsta dealer,youwouldwina handof thecardsleadto winninghands,andtheprobability of winningcanbe whatwe meanby theprobability of anevent that,if anexperiment is repeatedanin nitenumber of times,thefractionofexperiments in which theevent occursis itsprobability.

7 Ontheotherhand,thesubjectivistinterpreta tionisthata probability represents anindividualbeliefthata certainevent mostappropriatewhenexperiments cannotbe repeated,such as in which interpretationis used,thesameaxiomatictheoryis thischapter,we reviewsomeof thekey backgroundconceptsin probability ProbabilityA probability spaceis a triple( ;F; P) which is usedto describe theoutcomesof a is thesetof allpossibleelementaryexperiment outcomes!. ThesetFis a collectionof subsetsof which is ,ifAi2 F; i= 1; : : :, thenAi ;[1i=1Ai2F; Ai2F. (Thenotation Ais usedto denotethecomplement ofA, or A= A, whereB A=fxjx2Bx62Ag.)ThesetFis calleda - eldbecauseof itsclosureundercountableunion,andis referredto as thesetof Fis calledanevent.]

8 Notethattheabove propertiesalsoimplythatFis probability valuein [0;1] to each event containedinF; thatis,it mapsthesetof events into theclosedunitinterval [0;1].Furthermore,theprobability measurehassomeimportantproperties, eventsA,Baresaidto be mutuallyexclusive ifA\B=;, theempty aprobability ( )= (A) 0 ([1i=1Ai) =P1i=1P(Ai) ifAi\Aj=;foralli6=j. Thisproperty is calledthecountableadditivityproperty of theprobability properties, probability measurescanbe shownto satisfyadditionalproperties, (A) = 1 P( A). (;) = 0 (A[B) =P(A) +P(B) P(A\B).4. IfB A;thenP(B) P(A).Consideragaintheexampleof a shu eof a deck of outcomes;forexample,anevent may be thesetof allorderingssuch thattheaceofspadesis the is theconceptof event neededover andabove theconceptof outcome?]]

9 For experiments wherethesetof outcomesis discreteand nite,such as a shu eof a deck of cards,onecanconsideronlythesetofoutcomes ,as each outcomecanbe anindividualevent. However,therearemay situationswherewe wantto modelthesetof possibleoutcomesas continuous,ratherthandiscrete;forinstanc e,theexperiment ofpickinga randomnumber in theinterval [0,1].In such cases,it is oftenimpossibleto associatenon-zeroprobabilitieswithindivi dualoutcomes;indeed,thereareanuncountabl enumber of outcomes,andnoneoftheaxiomsof probability canbe usedto combinetheprobabilitiesof an uncountablenumber of probability , thereareat mosta nitenumber of mutuallyexclusive events which have probabilityof at least1=n. Thus,by de ningtheprobability measureonevents ratherthanonanuncountablenumberof outcomes,we canfocusourde nitiononthesigni cant outcomesof experiments,andalsoprovideameaningfulway of issueis thatnoteverysubsetof canbe anevent, becauseit is notpossibletoassigna probability to each subsetin a mannerwhich is consistent withtheaxiomsof probability instance,considerthefollowingconstructio nof a subsetof [0;1]:LetAs=fx2[0;1]jx ymod 1 is arationalnumber forallothery2 Asg.

10 ConstructthesetB=foneelement fromeach distinctAsg. Denotetheprobability measurePas theuniformmeasure,such thatP([a; b)) =b afor0 a b 1. Now, notethefollowingpropertiesof theconstructedsets:[sAs= [0;1]; As\At=;ifAs6= +r=fyjy=x+r; x2Bg. DenotebyBi=B+riforeach rationalnumberri. Notethefollowing:1. Therearea countablenumber \Bj=;ifi6=j, becauseBcontainsoneandonlyoneelement fromeachAs. Notethat,if theconclusionwerenottrue,thenthereis arex; y2B; x6=ysuch thatx+ri=y+rj, which wouldimplythatx; y2 Asforsomes, [1i=1Bi=[sAs= .Now,considerourdilemmaifBwereanevent, whatprobability wouldwe assigntoB? Clearly, byconstruction,P(Bi) =P(Bj) =P(B). So,ifP(B)6= 0, then,sincetheBiaremutuallydisjoint,P( )= (B) = 0, thenP( )= 0 also!]]]]