Search results with tag "1 discrete time markov chains"

1 Discrete-time Markov chains - Columbia University

3. Random walk: Let f n: n 1gdenote any iid sequence (called the increments), and de ne X n def= 1 + + n; X 0 = 0: (2) The Markov property follows since X n+1 = X n + n+1; n 0 which asserts that the future, given the present state, only depends on the present state X n and an independent (of the past) r.v. n+1. When P( = 1) = p;P( = 1) = 1 p, then the random walk is called a simple random

  University, Time, Chain, Discrete, Columbia university, Columbia, Markov, 1 discrete time markov chains