Transcription of Convolutional Sequence to Sequence Learning
{{id}} {{{paragraph}}}
Example: quiz answers
Convolutional Sequence to Sequence Learning
Convolutional Sequence to Sequence LearningJonas GehringMichael AuliDavid GrangierDenis YaratsYann N. DauphinFacebook AI ResearchAbstractThe prevalent approach to Sequence to sequencelearning maps an input Sequence to a variablelength output Sequence via recurrent neural net-works. We introduce an architecture based en-tirely on Convolutional neural to recurrent models, computations over allelements can be fully parallelized during trainingto better exploit the GPU hardware and optimiza-tion is easier since the number of non-linearitiesis fixed and independent of the input length. Ouruse of gated linear units eases gradient propaga-tion and we equip each decoder layer with a sep-arate attention module. We outperform the accu-racy of the deep LSTM setup of Wu et al.
works, e.g. we can obtain a feature representation captur-ing relationships within a window of n words by applying only O (n k) convolutional operations for kernels of width k , compared to a linear number O (n ) for recurrent neu-ral networks. Inputs to a convolutional network are fed through a constant number of kernels and non-linearities,
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: