Transcription of 1 IEOR 6711: Continuous-Time Markov Chains
{{id}} {{{paragraph}}}
Example: marketing
1 IEOR 6711: Continuous-Time Markov Chains
Copyrightc 2009 by Karl Sigman1 IEOR 6711 : Continuous-Time Markov ChainsA Markov chain in discrete time ,{Xn:n 0}, remains in any state for exactly oneunit of time before making a transition (change of state). We proceed now to relax thisrestriction by allowing a chain to spend a continuous amount of time in any state, butin such a way as to retain the Markov property. As motivation, suppose we consider therat in the open maze. Clearly it is more realistic to be able to keep track of where therat is at any continuous -timet 0 as oppposed to only where the rat is aftern steps .Assume throughout that our state space isS=Z={ , 2, 1,0,1,2, }(or somesubset thereof). Suppose now that whenever a chain enters statei S, independent ofthe past, the length of time spent in stateiis a continuous , strictly positive (and proper)random variableHicalled theholding timein statei.
with a continuous-time stochastic process fX(t) : t 0gwith state space S. Our objective is to place conditions on the holding times to ensure that the continuous-time process satis es the Markov property: The future, fX(s+ t) : t 0g, given the present state, X(s), is independent of the past, fX(u) : 0 u<sg. Such a process will
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: