Lecture 2: Markov Decision Processes - David Silver

Lecture 2: Markov Decision ProcessesLecture 2: Markov Decision ProcessesDavid SilverLecture 2: Markov Decision Processes1 Markov Processes2 Markov Reward Processes3 Markov Decision Processes4 Extensions to MDPsLecture 2: Markov Decision ProcessesMarkov ProcessesIntroductionIntroduction to MDPsMarkov Decision processesformally describe an environmentfor reinforcement learningWhere the environment isfully The currentstatecompletely characterises the processAlmost all RL problems can be formalised as MDPs, control primarily deals with continuous MDPsPartially observable problems can be converted into MDPsBandits are MDPs with one stateLecture 2: Markov Decision ProcessesMarkov ProcessesMarkov PropertyMarkov Property The future is independent of the past given the present DefinitionA stateStisMarkovif and only ifP[St+1|St] =P[St+1|S1.]

State Transition Matrix For a Markov state s and successor state s0, the state transition probability is de ned by P ss0= P S t+1 = s 0jS t = s State transition matrix Pde nes transition probabilities from all states s to all successor states s0, to P = from 2 6 4 P 11::: P 1n... P n1::: P nn 3 7 5 where each row of the matrix sums to 1.

Tags:

  Transition, Probabilities, Transition probabilities

Information

Related search queries