Chapter 1 Markov Chains

Chapter 1 Markov ChainsA sequence of random variablesX0,X1,..with values in a countable setSisa Markov chain if at any timen, the future states (or values)Xn+1,Xn+2,..depend on the historyX0,..,Xnonly through the present are fundamental stochastic processes that have many diverse applica-tions. This is because a Markov chain represents any dynamical system whosestates satisfy the recursionXn=f(Xn 1,Yn),n 1, whereY1, and identically distributed ( ) andfis a deterministic func-tion. That is, the new stateXnis simply a function of the last state andan auxiliary random variable. Such system dynamics are typical of those forqueue lengths in call centers, stresses on materials, waiting times in produc-tion and service facilities, inventories in supply Chains , parallel-processingsoftware, water levels in dams, insurance funds, stock prices, Chapter begins by describing the basic structure of a Markov chainand how its single-step transition probabilities determine its evolution. For in-stance, what is the probability of reaching a certain state, and how long doesit take to reach it?

2 1MarkovChains 1.1 Introduction This section introduces Markov chains and describes a few examples. A discrete-time stochastic process {X n: n ≥ 0} on a countable set S is a collection of S-valued random variables defined on a probability space (Ω,F,P).The Pis a probability measure on a family of events F (a σ-field) in an event-space Ω.1 The set Sis the state space of the process, and the

Loading..

Tags:

  Chain

Information

Related search queries