Transcription of 1 Discrete-time Markov chains
{{id}} {{{paragraph}}}
Example: confidence
1 Discrete-time Markov chains
Copyrightc 2009 by Karl Sigman1 Discrete-time Markov Stochastic processes in discrete timeAstochastic processin discrete timen IN ={0,1,2,..}is a sequence of random variables(rvs)X0,X1,X2,..denoted byX={Xn:n 0}(or justX={Xn}). We refer to the valueXnas thestateof the process at timen, withX0denoting the initial state. If the randomvariables take values in a discrete space such as the integers ZZ ={.., 2, 1,0,1,2,..}(orsome subset of them), then the stochastic process is said to be discrete-valued; we then denotethe states byi,jand so on. In general, however, the collection of possible values that theXncan take on is called thestate space, is denoted bySand could be, for example,d dimensionalEuclidean space IRd, d 1, or a subset of processes are meant to model the evolution over time of real phenomena forwhich randomness is inherent.
State 0 here is an example of an absorbing state: Whenever the chain enters state 0, it remains in that state forever after; P(X n+1 = 0 jX n = 0) = P 00 = 1. Of interest is
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: