Hamilton-Jacobi-Bellman Equation

Hamilton-Jacobi-Bellman EquationFeb 25, 2008 What is it?The Hamilton-Jacobi-Bellman (HJB) Equation is the continuous-time analog to the discrete deterministic dynamic programming algorithm Discrete VS Continuous xk 1=f xk,uk x t =f x t ,u t k 0,..,N0 t TJN xN =gN xN V T,x =h x Jk xk =minuk Uk{gk xk,uk +Jk 1 xk,uk }0=minu U{g x,u tV t,x + xV t,x 'f x,u }h x T 0Tg x t ,u t dtgN xN k=0N 1gk xk,uk HJB Equation Extension of hamilton -Jacobi Equation (classical mechanics) Solution is the optimal cost-to-go function Applications path planning medical financialDerivation Start with continuous time intervalt [0,T] Discretize into N pieces so that =TN Denotexk=x k , k=0,..,Nuk=u k , k=0,..,NDerivation Remember x t =f x t ,u t Approximate the continuous time by xk 1=xk f xk,uk h xN k=0N 1 g xk,uk h x T 0Tg x t ,u t dtDerivationJ* t,x J* t,x : Optimal cost-to-go function for continuous time problem: Optimal cost-to-go function for discrete time approximationDerivation From discrete time DPJN xN =gN xN Jk xk =minuk Uk{gk xk,uk +Jk 1 xk,uk } For the discrete time approximation J* N ,x =h x J* k ,x =minuk Uk{ g x,u J* k 1 ,x f x,u }Derivation Assume that the Taylor series expans

The Hamilton-Jacobi-Bellman (HJB) equation is the continuous-time analog to the discrete deterministic dynamic programming algorithm. Discrete VS Continuous xk 1= f ...

Loading..

Tags:

  Hamilton

Information

Related search queries