Transcription of Language Models as Knowledge Bases?
{{id}} {{{paragraph}}}
Language Models as Knowledge Bases? Fabio Petroni1 Tim Rockta schel1,2 Patrick Lewis1,2 Anton Bakhtin1. Yuxiang Wu1,2 Alexander H. Miller1 Sebastian Riedel1,2. 1. Facebook AI Research 2. University College London {fabiopetroni, rockt, plewis, yolo, yuxiangwu, ahm, Abstract Memory Query Answer (Dante, born-in, X). Recent progress in pretraining Language mod- els on large textual corpora led to a surge Symbolic KB. KB Dante Florence Memory Access of improvements for downstream NLP tasks. born-in Whilst learning linguistic Knowledge , these Florence Models may also be storing relational knowl- edge present in the training data, and may Dante was born in [Mask].. be able to answer queries structured as fill- Neural LM. LM Florence in-the-blank cloze statements. Language Memory Access Models have many advantages over structured ELMo/BERT. Knowledge bases: they require no schema en- gineering, allow practitioners to query about Figure 1: Querying Knowledge bases (KB) and lan- an open class of relations, are easy to extend to guage Models (LM) for factual Knowledge .}
knowledge bases: they require no schema en-gineering, allow practitioners to query about an open class of relations, are easy to extend to more data, and require no human supervision to train. We present an in-depth analysis of the relational knowledge already present (without fine-tuning) in a wide range of state-of-the-art pretrained ...
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
{{id}} {{{paragraph}}}