Transcription of Deep Contextualized Word Representations
{{id}} {{{paragraph}}}
Proceedings of NAACL-HLT 2018, pages 2227 2237 New Orleans, Louisiana, June 1 - 6, 2018 Association for Computational LinguisticsDeep Contextualized word representationsMatthew E. Peters , Mark Neumann , Mohit Iyyer , Matt Gardner Clark , Kenton Lee , Luke Zettlemoyer Allen Institute for Artificial Intelligence Paul G. Allen School of Computer Science & Engineering, University of WashingtonAbstractWe introduce a new type ofdeep contextual-izedword representation that models both (1)complex characteristics of word use ( , syn-tax and semantics), and (2) how these usesvary across linguistic contexts ( , to modelpolysemy). Our word vectors are learned func-tions of the internal states of a deep bidirec-tional language model (biLM), which is pre-trained on a large text corpus.
pivot word itself in the representation and are computed with the encoder of either a supervised neural machine translation (MT) system (CoVe; McCann et al. , 2017 ) or an unsupervised lan-guage model ( Peters et al. , 2017 ). Both of these approaches beneÞt from large datasets, although the MT approach is limited by the size of parallel corpora.
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
{{id}} {{{paragraph}}}