Convolutional Sequence to Sequence Learning - arXiv
Convolutional Sequence to Sequence LearningJonas GehringMichael AuliDavid GrangierDenis YaratsYann N. DauphinFacebook AI ResearchAbstractThe prevalent approach to Sequence to sequencelearning maps an input Sequence to a variablelength output Sequence via recurrent neural net-works. We introduce an architecture based en-tirely on Convolutional neural to recurrent models, computations over allelements can be fully parallelized during trainingto better exploit the GPU hardware and optimiza-tion is easier since the number of non-linearitiesis fixed and independent of the input length. Ouruse of gated linear units eases gradient propaga-tion and we equip each decoder layer with a sep-arate attention module. We outperform the accu-racy of the deep LSTM setup of Wu et al. (2016)on both WMT 14 English-German and WMT 14English-French translation at an order of magni-tude faster speed, both on GPU and IntroductionSequence to Sequence Learning has been successful inmany tasks such as machine translation, speech recogni-tion (Sutskever et al.)
by the decoder network to yield output element represen-tations that are being fed back into the decoder network g = ( g1;:::;gn). Position embeddings are useful in our architecture since they give our model a sense of which portion of the sequence in the input or output it is currently dealing with ( x5.4). 3.2. Convolutional Block Structure
Download Convolutional Sequence to Sequence Learning - arXiv
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: