Learning Phrase Representations using RNN Encoder- …

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724 1734,October 25-29, 2014, Doha, 2014 Association for Computational LinguisticsLearning Phrase Representations using RNN Encoder Decoderfor Statistical Machine TranslationKyunghyun ChoBart van Merri enboer Caglar GulcehreUniversit e de Montr BahdanauJacobs University, Bougares Holger SchwenkUniversit e du Maine, BengioUniversit e de Montr eal, CIFAR Senior this paper, we propose a novel neu-ral network model called RNN Encoder Decoder that consists of two recurrentneural networks (RNN). One RNN en-codes a sequence of symbols into a fixed-length vector representation, and the otherdecodes the representation into another se-quence of symbols. The encoder and de-coder of the proposed model are jointlytrained to maximize the conditional prob-ability of a target sequence given a sourcesequence.

date gates, each hidden unit will learn to capture dependencies over different time scales. Those unitsthatlearntocaptureshort-termdependencies will tend to have reset gates that are frequently ac-tive, but those that capture longer-term dependen-cies will have update gates that are mostly active. In our preliminary experiments, we found that

Loading..

Tags:

  Using, Gate

Information

Related search queries