Sequence to Sequence Learning with ... - Stanford University

[ ] 14 Dec 2014 Sequence to Sequence Learningwith Neural NetworksIlya V. Neural Networks (DNNs) are powerful models that have achieved excel-lent performance on difficult Learning tasks. Although DNNswork well wheneverlarge labeled training sets are available, they cannot be used to map sequences tosequences. In this paper, we present a general end-to-end approach to sequencelearning that makes minimal assumptions on the Sequence structure. Our methoduses a multilayered Long Short-Term Memory (LSTM) to map theinput sequenceto a vector of a fixed dimensionality, and then another deep LSTM to decode thetarget Sequence from the vector. Our main result is that on anEnglish to Frenchtranslation task from the WMT 14 dataset, the translationsproduced by the LSTM achieve a BLEU score of on the entire test set, where the LSTM s BLEU score was penalized on out-of-vocabulary words.

1 Introduction Deep Neural Networks (DNNs) are extremely powerful machine learning models that achieve ex-cellent performanceon difficult problems such as speech rec ognition[13, 7] and visual object recog-nition [19, 6, 21, 20]. DNNs are powerful because they can perform arbitrary parallel computation for a modest number of steps.

Loading..

Tags:

  Introduction, Machine, Learning, Sequence, Machine learning, Stanford, Sequence to sequence learning

Information

Related search queries