Transcription of Abstract - arXiv
{{id}} {{{paragraph}}}
Example: bankruptcy
Abstract - arXiv
Offline Reinforcement Learning as One BigSequence Modeling ProblemMichael JannerQiyang LiSergey LevineUniversity of California at Berkeley{janner, learning (RL) is typically concerned with estimating stationarypolicies or single-step models, leveraging the Markov property to factorize prob-lems in time. However, we can also view RL as a generic sequence modelingproblem, with the goal being to produce a sequence of actions that leads to asequence of high rewards. Viewed in this way, it is tempting to consider whetherhigh-capacity sequence prediction models that work well in other domains, suchas natural-language processing, can also provide effective solutions to the RLproblem. To this end, we explore how RL can be tackled with the tools of sequencemodeling, using a Transformer architecture to model distributions over trajectoriesand repurposing beam search as a planning algorithm.}
Algorithm 1 Beam search 1: Require Input sequence x, vocabulary V, sequence length T, beam width B 2: Initialize Y 0 = f( ) g 3: for t= 1;:::;Tdo 4: C t f y t 1 yj t 1 2Y t 1 and 2Vg // candidate single-token extensions 5: Y t argmax Y Ct;jYj=B logP (Y jx) // B most likely sequences from candidates 6: end for 7: Return argmax y2Y T logP (y jx) We investigate two simple …
Loading..
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: