Transcription of 1 Limiting distribution for a Markov chain
{{id}} {{{paragraph}}}
Example: quiz answers
1 Limiting distribution for a Markov chain
Copyrightc 2009 by Karl Sigman1 Limiting distribution for a Markov chainIn these Lecture Notes, we shall study the Limiting behavior of Markov chains as timen .In particular, under suitable easy-to-check conditions, we will see that a Markov chain possessesa Limiting probability distribution , = ( j)j S, and that the chain , if started off initially withsuch a distribution will be a stationary stochastic process. We will also see that we can find by merely solving a set of linear Communication classes and irreducibility for Markov chainsFor a Markov chain with state spaceS, consider a pair of states (i,j). We say thatjis reachablefromi, denoted byi j, if there exists an integern 0 such thatPnij>0. This means thatstarting in statei, there is a positive probability (but not necessarily equal to 1) that the chainwill be in statejat timen(that is,nsteps later);P(Xn=j|X0=i)>0. Ifjis reachablefromi, andiis reachable fromj, then the statesiandjare said tocommunicate, denoted byi j.
A recurrent state jis called positive recurrent if the expected amount of time to return to state jgiven that the chain started in state jhas nite rst moment: E(˝ jj) <1: A recurrent state jfor which E(˝ jj) = 1is called null recurrent. In general even for i6= j, we de ne ˝ …
Loading..
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: