Dual Graph Convolutional Networks for Aspect-based ...

1 Proceedings of the 59th Annual Meeting of the Association for Computational Linguisticsand the 11th International Joint Conference on Natural Language Processing, pages 6319 6329 August 1 6, 2021. 2021 Association for Computational Linguistics6319 Dual Graph Convolutional Networks for Aspect-based Sentiment AnalysisRuifan Li1 , Hao Chen1, Fangxiang Feng1,Zhanyu Ma1, Xiaojie WANG1, and Eduard Hovy21 School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China2 Language Technologies Institute, Carnegie Mellon University, USA{rfli, ccchenhao997, fxfeng, mazhanyu, sentiment analysis is a fine-grained sentiment classification , Graph neural Networks over depen-dency trees have been explored to explicitlymodel connections between aspects and opin-ion words. However, the improvement is lim-ited due to the inaccuracy of the dependencyparsing results and the informal expressionsand complexity of online reviews.}

2 To over-come these challenges, in this paper, we pro-pose a dual Graph Convolutional Networks (Du-alGCN) model that considers the complemen-tarity of syntax structures and semantic cor-relations simultaneously. Particularly, to al-leviate dependency parsing errors, we designa SynGCN module with rich syntactic knowl-edge. To capture semantic correlations, wedesign a SemGCN module with , we propose or-thogonal and differential regularizers to cap-ture semantic correlations between words pre-cisely by constraining attention scores in theSemGCN orthogonal regular-izer encourages the SemGCN to learn seman-tically correlated words with less overlap foreach word. The differential regularizer encour-ages the SemGCN to learn semantic featuresthat the SynGCN fails to capture. Experimen-tal results on three public datasets show thatour DualGCN model outperforms state-of-the-art methods and verify the effectiveness of IntroductionSentiment analysis has become a popular topic innatural language processing (Liu, 2012; Li andHovy, 2017).

3 Aspect-based sentiment analysis(ABSA) talks an entity-level oriented fine-grainedsentiment analysis task that aims to determine sen-timent polarities of given aspects in a sentence. In Corresponding 1: An example sentence with its dependencytree from the restaurant reviews. This sentence containstwo aspects but with opposite sentiment 1, the comment is about a restaurant sentiment polarity of the two aspects price and service are positive and negative, respec-tively. Thus, ABSA can precisely identify user sattitudes towards a certain aspect, rather than sim-ply assigning a sentiment polarity for a key point in solving the ABSA task is tomodel the dependency relationship between an as-pect and its corresponding opinion , there probably exist multiple aspectsand different opinion expressions in a sentence. Tojudge the sentiment of a particular aspect, previousstudies (Wang et al.)

4 , 2016; Tang et al., 2016a; Maet al., 2017; Chen et al., 2017; Fan et al., 2018;Huang et al., 2018; Gu et al., 2018) have proposedvarious recurrent neural Networks (RNNs) with at-tention mechanisms to generate aspect-specific sen-tence representations and have achieved appealingresults. However, an inherent defect makes theattention mechanism vulnerable to noise in the sen-tence. Take Figure 1 as an example; for the aspect service , the opinion word reasonable may re-ceive more attention than the opinion word poor .However, the reasonable refers to another as-pect, , price .More recent efforts (Zhang et al., 2019; Sunet al., 2019b; Huang and Carley, 2019; Zhang andQian, 2020; Chen et al., 2020; Liang et al., 2020;Wang et al., 2020; Tang et al., 2020) have been de-6320voted to Graph Convolutional Networks (GCNs) andgraph attention Networks (GATs) over dependencytrees, which explicitly exploit the syntactic struc-ture of a sentence.

5 Consider the dependency treein Figure 1; the syntactic dependency can establishconnections between the words in a sentence. Forexample, a dependency relation exists between theaspect price and the opinion word reasonable .However, two challenges arise when applying syn-tactic dependency knowledge to the ABSA task: 1)the inaccuracy of the dependency parsing resultsand 2) GCNs over dependency trees do not workwell as expected on datasets that are not sensitiveto syntactic dependency due to the informal expres-sion and complexity of online this paper, we propose a novel architecture,the dual Graph convolution network (DualGCN), asshown in Figure 2, to solve the aforementionedchallenges. For the first challenge, we use theprobability matrix of all dependency arcs from adependency parser to build a syntax-based graphconvolutional network (SynGCN).

6 The idea behindthis approach is that the probability matrix rep-resenting dependencies between words containsrich syntactic information compared with the finaldiscrete output of a dependency parser. For thesecond, we construct a semantic correlation-basedgraph Convolutional network (SemGCN) by utiliz-ing a self-attention mechanism. The idea behindthis approach is that the attention matrix shapedby self-attending, also viewed as an edge-weighteddirected Graph , can represent semantic correlationsbetween words. Moreover, motivated by the workof DGEDT (Tang et al., 2020), we utilize a BiAffinemodule to bridge relevant information between theSynGCN and SemGCN , we design two regularizers to en-hance our DualGCN model. We observe that thesemantically related terms of each word should notoverlap. Therefore, we encourage the attentionprobability distributions over words to be orthog-onal.

7 To this end, we incorporate an orthogonalregularizer on the attention probability matrix forthe SemGCN module. Moreover, the two represen-tations learned from the SynGCN and SemGCNmodules should contain significantly distinct infor-mation captured by the syntactic dependency andthe semantic correlation. Therefore, we expect thatthe SemGCN module could learn semantic repre-sentations different from syntactic , we propose a differential regularizer betweenthe SynGCN and SemGCN contributions are highlighted as follows: We propose a DualGCN model for the ABSA task. Our DualGCN considers both the syntacticstructure and the semantic correlation within agiven sentence. Specifically, our DualGCN in-tegrates the SynGCN and SemGCN networksthrough a mutual BiAffine module. We propose orthogonal and differential regular-izers. The orthogonal regularizer encouragesthe SemGCN network to learn an orthogonal se-mantic attention matrix, whereas the differentialregularizer encourages the SemGCN network tolearn semantic features distinct from the syntac-tic ones built from the SynGCN network.

8 We conduct extensive experiments on the Se-mEval 2014 and Twitter datasets. The experi-mental results demonstrate the effectiveness ofour DualGCN model. Additionally, the sourcecode and preprocessed datasets used in our workare provided on Related WorkTraditional sentiment analysis tasks are sentence-level or document-level contrast,ABSA is an entity-level oriented and a more fine-grained task for sentiment analysis. Earlier meth-ods (Titov and McDonald, 2008; Jiang et al., 2011;Kiritchenko et al., 2014; Vo and Zhang, 2015) areusually based on handcrafted features and fail tomodel the dependency between the given aspectand its , various attention-based neural net-works have been proposed to implicitly model thesemantic relation of an aspect and its context to cap-ture the opinion expression component (Wang et al.,2016; Tang et al.)

9 , 2016a,b; Ma et al., 2017; Chenet al., 2017; Fan et al., 2018; Huang et al., 2018; Guet al., 2018; Li et al., 2018a; Tan et al., 2019). Forinstance, (Wang et al., 2016) proposed attention-based LSTMs for aspect-level sentiment classifica-tion. (Tang et al., 2016b) and (Chen et al., 2017)both introduced a hierarchical attention network toidentify important sentiment information relatedto the given aspect. (Fan et al., 2018) exploited amulti-grained attention mechanism to capture theword-level interaction between aspects and theircontext. (Tan et al., 2019) designed a dual attention1 to recognize conflicting opinions. In addi-tion, the pre-trained language model BERT (Devlinet al., 2019) has achieved remarkable performancein many NLP tasks, including ABSA. (Sun et al.,2019a) transformed ABSA task into a sentence pairclassification task by constructing an auxiliary sen-tence.

10 (Xu et al., 2019) proposed a post-trainingapproach on the BERT to enhance the performanceof fine-tuning stage for the ABSA trend explicitly leverages syntacticknowledge. This type of knowledge helps to es-tablish connections between the aspects and theother words in a sentence to learn syntax-awarefeature representations of aspects. (Dong et al.,2014) proposed a recursive neural network to adap-tively propagate the sentiment of words to the as-pect along the dependency tree. (He et al., 2018)introduced an attention model that incorporatedsyntactic information to compute attention weights.(Phan and Ogunbona, 2020) utilized the syntacticrelative distance to reduce the impact of this line, a few works extend the GCNand GAT models by means of a syntactical depen-dency tree and develop several outstanding mod-els (Zhang et al., 2019; Sun et al.)