Transcription of Convolutional Sequence to Sequence Learning - arXiv
{{id}} {{{paragraph}}}
Example: marketing
Convolutional Sequence to Sequence Learning - arXiv
Convolutional Sequence to Sequence LearningJonas GehringMichael AuliDavid GrangierDenis YaratsYann N. DauphinFacebook AI ResearchAbstractThe prevalent approach to Sequence to sequencelearning maps an input Sequence to a variablelength output Sequence via recurrent neural net-works. We introduce an architecture based en-tirely on Convolutional neural to recurrent models, computations over allelements can be fully parallelized during trainingto better exploit the GPU hardware and optimiza-tion is easier since the number of non-linearitiesis fixed and independent of the input length. Ouruse of gated linear units eases gradient propaga-tion and we equip each decoder layer with a sep-arate attention module. We outperform the accu-racy of the deep LSTM setup of Wu et al. (2016)on both WMT 14 English-German and WMT 14 English-French translation at an order of magni-tude faster speed, both on GPU and IntroductionSequence to Sequence Learning has been successful inmany tasks such as machine translation, speech recogni-tion (Sutskever et al.)
Dauphin et al.,2016) as non-linearity which implement a simple gating mechanism over the output of the convolu-tion Y = [ A B ] 2 R 2d: v([A B ]) = A (B ) where A;B 2 R d are the inputs to the non-linearity, is the point-wise multiplication and the output v([A B ]) 2 R d is half the size of Y . The gates (B ) control which
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: