Maximum Entropy Inverse Reinforcement Learning

← Back to document page

Maximum Entropy Inverse Reinforcement LearningBrianD. Ziebart, Andrew Maas, Bagnell,andAnind K. DeySchool of Computer ScienceCarnegie Mellon UniversityPittsburgh, PA research has shown the benefit of framing problemsof imitation Learning as solutions to Markov Decision Prob-lems. This approach reduces Learning to the problem of re-covering a utility function that makes the behavior inducedby a near-optimal policy closely mimic demonstrated behav-ior. In this work, we develop a probabilistic approach basedon the principle of Maximum Entropy . Our approach providesa well-defined, globally normalized distribution over decisionsequences, while providing the same performance guaranteesas existing develop our technique in the context of modeling real-world navigation and driving behaviors where collected datais inherently noisy and imperfect.

sition distribution, T. Paths in these MDPs (Figure 1d) are now determined by the action choices of the agent and the random outcomes of the MDP. Our distribution over paths must take this randomness into account. We use the maximum entropy distribution of paths con-ditioned on the transition distribution, T, and constrained to

  Distribution, Learning, Maximum, Reinforcement, Inverse, Entropy, Maximum entropy inverse reinforcement learning