A Unified MRC Framework for Named Entity Recognition

1 Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5849 5859 July 5 - 10, 2020 Association for Computational Linguistics5849A Unified MRC Framework for Named Entity RecognitionXiaoya Li , Jingrong Feng , Yuxian Meng , Qinghong Han , Fei Wu and Jiwei Li Department of Computer Science and Technology, Zhejiang University {xiaoyali, jingrongfeng, task of Named Entity Recognition (NER)is normally divided into nested NER and flatNER depending on whether Named entities arenested or not. Models are usually separatelydeveloped for the two tasks, since sequence la-beling models are only able to assign a singlelabel to a particular token, which is unsuitablefor nested NER where a token may be assignedseveral this paper, we propose a Unified frameworkthat is capable of handling both flat and nestedNER tasks.}

2 Instead of treating the task of NERas a sequence labeling problem, we propose toformulate it as a machine reading comprehen-sion (MRC) task. For example, extracting en-tities with thePER(PERSON) label is formal-ized as extracting answer spans to the question which person is mentioned in the text .Thisformulation naturally tackles the Entity over-lapping issue in nested NER: the extractionof two overlapping entities with different cat-egories requires answering two independentquestions. Additionally, since the query en-codes informative prior knowledge, this strat-egy facilitates the process of Entity extraction,leading to better performances for not onlynested NER, but flat conduct experiments on both nestedand flat NER re-sults demonstrate the effectiveness of theproposed are able toachieve a vast amount of performance boostover current SOTA models on nested NERdatasets, , + , + , + , + ,re-spectively on ACE04, ACE05, GENIA andKBP17, as well as flat NER datasets, ,+ , + , + , + respectively onEnglish CoNLL 2003, English.

3 Chinese MSRA and Chinese code and datasets can be 1: Examples fornestedentities from GENIAand ACE04 IntroductionNamed Entity Recognition (NER) refers to thetask of detecting the span and the semantic cate-gory of entities from a chunk of text. The task canbe further divided into two sub-categories, nestedNER and flat NER, depending on whether entitiesare nested or not. Nested NER refers to a phe-nomenon that the spans of entities (mentions) arenested, as shown in Figure 1. Entity overlappingis a fairly common phenomenon in natural task of flat NER is commonly formalizedas a sequence labeling task: a sequence labelingmodel (Chiu and Nichols, 2016; Ma and Hovy,2016; Devlin et al.)

4 , 2018) is trained to assigna single tagging class to each unit within a se-quence of tokens. This formulation is unfortu-nately incapable of handling overlapping entitiesin nested NER (Huang et al., 2015; Chiu andNichols, 2015), where multiple categories need tobe assigned to a single token if the token partic-ipates in multiple entities. Many attempts havebeen made to reconcile sequence labeling modelswith nested NER (Alex et al., 2007; Byrne, 2007;Finkel and Manning, 2009; Lu and Roth, 2015;Katiyar and Cardie, 2018), mostly based on thepipelined systems. However, pipelined systemssuffer from the disadvantages of error propagation,long running time and the intensiveness in devel-oping hand-crafted features, by the current trend of formalizing5850 NLP problems as question answering tasks (Levyet al.

5 , 2017; McCann et al., 2018; Li et al., 2019),we propose a new Framework that is capable ofhandling both flat and nested NER. Instead oftreating the task of NER as a sequence labelingproblem, we propose to formulate it as a SQuAD-style (Rajpurkar et al., 2016, 2018) machine read-ing comprehension (MRC) task. Each Entity typeis characterized by a natural language query, andentities are extracted by answering these queriesgiven the contexts. For example, the task of as-signing the PER(PERSON) label to [Washington]was born into slavery on the farm of James Bur-roughs is formalized as answering the question which person is mentioned in the text?

6 Thisstrategy naturally tackles the Entity overlapping is-sue in nested NER: the extraction of two entitieswith different categories that overlap requires an-swering two independent MRC formulation also comes with anotherkey advantage over the sequence labeling formu-lation. For the latter, golden NER categories aremerely class indexes and lack for semantic priorinformation for Entity categories. For example, theORG(ORGANIZATION) class is treated as a one-hot vector in sequence labeling training. This lackof clarity on what to extract leads to inferior per-formances. On the contrary, for the MRC formu-lation, the query encodes significant prior infor-mation about the Entity category to extract.

7 Forexample, the query find an organization such ascompany, agency and institution in the context encourages the model to link the word organi-zation in the query to location entities in thecontext. Additionally, by encoding comprehen-sive descriptions ( , company, agency and in-stitution ) of tagging categories ( ,ORG), themodel has the potential to disambiguate similartagging conduct experiments on bothnestedandflatNER datasets to show the generality of our ap-proach. Experimental results demonstrate its ef-fectiveness. We are able to achieve a vast amountof performance boost over current SOTA modelson nested NER datasets, , + , + , + ,+ , respectively on ACE04, ACE05, GENIAand KBP17, as well as flat NER datasets, ,+ , + , + , + respectively on En-glish CoNLL 2003, English OntoNotes , Chi-nese MSRA, Chinese OntoNotes We wishthat our work would inspire the introduction ofnew paradigms for the Entity Recognition Related Named Entity Recognition (NER)Traditional sequence labeling models use CRFs(Lafferty et al.)

8 , 2001; Sutton et al., 2007) as abackbone for NER. The first work using neuralmodels for NER goes back to 2003, when Ham-merton (2003) attempted to solve the problem us-ing unidirectional LSTMs. Collobert et al. (2011)presented a CNN-CRF structure, augmented withcharacter embeddings by Santos and Guimaraes(2015).Lample et al. (2016) explored neuralstructures for NER, in which the bidirectionalLSTMs are combined with CRFs with featuresbased on character-based word representationsand unsupervised word representations. Ma andHovy (2016) and Chiu and Nichols (2016) useda character CNN to extract features from charac-ters. Recent large-scale language model pretrain-ing methods such as BERT (Devlin et al.

9 , 2018)and ELMo (Peters et al., 2018a) further enhancedthe performance of NER, yielding Nested Named Entity RecognitionThe overlapping between entities (mentions) wasfirst noticed by Kim et al. (2003), who developedhandcrafted rules to identify overlapping men-tions. Alex et al. (2007) proposed two multi-layerCRF models for nested NER. The first model isthe inside-out model, in which the first CRF identi-fies the innermost entities, and the successive layerCRF is built over words and the innermost enti-ties extracted from the previous CRF to identifysecond-level entities, etc. The other is the outside-in model, in which the first CRF identifies out-ermost entities, and then successive CRFs wouldidentify increasingly nested entities.

10 Finkel andManning (2009) built a model to extract nested en-tity mentions based on parse trees. They made theassumption that one mention is fully contained bythe other when they overlap. Lu and Roth (2015)proposed to use mention hyper-graphs for recog-nizing overlapping mentions. Xu et al. (2017) uti-lized a local classifier that runs on every possi-ble span to detect overlapping mentions and Kati-yar and Cardie (2018) used neural models to learnthe hyper-graph representations for nested et al. (2018) dynamically stacked flatNER layers in a hierarchical manner. Lin et (2019a) proposed the Anchor-Region Networks(ARNs) architecture by modeling and leveragingthe head-driven phrase structures of nested entitymentions.