Transcription of Deep Reinforcement Learning with Double Q-learning
{{id}} {{{paragraph}}}
Example: air traffic controller
Deep Reinforcement Learning with Double Q-learning
Deep Reinforcement Learning with Double Q-learningHado van HasseltandArthur GuezandDavid SilverGoogle DeepMindAbstractThe popular Q- Learning algorithm is known to overestimateaction values under certain conditions. It was not previouslyknown whether, in practice, such overestimations are com-mon, whether they harm performance, and whether they cangenerally be prevented. In this paper, we answer all thesequestions affirmatively. In particular, we first show that therecent DQN algorithm, which combines Q- Learning with adeep neural network, suffers from substantial overestimationsin some games in the Atari 2600 domain. We then show thatthe idea behind the Double Q- Learning algorithm, which wasintroduced in a tabular setting, can be generalized to workwith large-scale function approximation. We propose a spe-cific adaptation to the DQN algorithm and show that the re-sulting algorithm not only reduces the observed overestima-tions, as hypothesized, but that this also leads to much betterperformance on several goal of Reinforcement Learning (Sutton and Barto, 1998)is to learn good policies for sequential decision problems,by optimizing a cumulative future reward signal.
tion. This is the idea behind Double Q-learning (van Hasselt, 2010). In the original Double Q-learning algorithm, two value functions are learned by assigning each experience ran-domly to update one of the two value functions, such that there are two sets of …
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: