Transcription of ERNIE: Enhanced Language Representation with Informative ...
{{id}} {{{paragraph}}}
Example: barber
ERNIE: Enhanced Language Representation with Informative ...
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1441 1451 Florence, Italy, July 28 - August 2, 2019 Association for Computational Linguistics1441 ERNIE: Enhanced Language Representation with Informative EntitiesZhengyan Zhang1,2,3 , Xu Han1,2,3 , Zhiyuan Liu1,2,3 , Xin Jiang4, Maosong Sun1,2,3, Qun Liu41 Department of Computer Science and Technology, Tsinghua University, Beijing, China2 Institute for Artificial Intelligence, Tsinghua University, Beijing, China3 State Key Lab on Intelligent Technology and Systems, Tsinghua University, Beijing, China4 Huawei Noah s Ark Language Representation models suchas BERT pre-trained on large-scale corporacan well capture rich semantic patterns fromplain text, and be fine-tuned to consistently im-prove the performance of various NLP , the existing pre-trained languagemodels rarely consider incorporating knowl-edge graphs (KGs), which can provide richstructured knowledge facts for better languageunderstanding.
pora and KGs to train an enhanced language rep-resentation model based on BERT. 3 Methodology In this section, we present the overall framework of ERNIE and its detailed implementation, includ-ing the model architecture in Section3.2, the novel pre-training task designed for encoding informa-tive entities and fusing heterogeneous information
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: