Dual Graph Convolutional Networks for Aspect-based ...

1 Proceedings of the 59th Annual Meeting of the Association for Computational Linguisticsand the 11th International Joint Conference on Natural Language Processing, pages 6319 6329 August 1 6, 2021. 2021 Association for Computational Linguistics6319 Dual Graph Convolutional Networks for Aspect-based sentiment AnalysisRuifan Li1 , Hao Chen1, Fangxiang Feng1,Zhanyu Ma1, Xiaojie WANG1, and Eduard Hovy21 School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China2 Language Technologies Institute, Carnegie Mellon University, USA{rfli, ccchenhao997, fxfeng, mazhanyu, sentiment analysis is a fine-grained sentiment classification , Graph neural Networks over depen-dency trees have been explored to explicitlymodel connections between aspects and opin-ion words.}

2 However, the improvement is lim-ited due to the inaccuracy of the dependencyparsing results and the informal expressionsand complexity of online reviews. To over-come these challenges, in this paper, we pro-pose a dual Graph Convolutional Networks (Du-alGCN) model that considers the complemen-tarity of syntax structures and semantic cor-relations simultaneously. Particularly, to al-leviate dependency parsing errors, we designa SynGCN module with rich syntactic knowl-edge. To capture semantic correlations, wedesign a SemGCN module with , we propose or-thogonal and differential regularizers to cap-ture semantic correlations between words pre-cisely by constraining attention scores in theSemGCN orthogonal regular-izer encourages the SemGCN to learn seman-tically correlated words with less overlap foreach word.

3 The differential regularizer encour-ages the SemGCN to learn semantic featuresthat the SynGCN fails to capture. Experimen-tal results on three public datasets show thatour DualGCN model outperforms state-of-the-art methods and verify the effectiveness of IntroductionSentiment analysis has become a popular topic innatural language processing (Liu, 2012; Li andHovy, 2017). Aspect-based sentiment analysis (ABSA) talks an entity-level oriented fine-grainedsentiment analysis task that aims to determine sen-timent polarities of given aspects in a sentence. In Corresponding 1: An example sentence with its dependencytree from the restaurant reviews.

4 This sentence containstwo aspects but with opposite sentiment 1, the comment is about a restaurant sentiment polarity of the two aspects price and service are positive and negative, respec-tively. Thus, ABSA can precisely identify user sattitudes towards a certain aspect, rather than sim-ply assigning a sentiment polarity for a key point in solving the ABSA task is tomodel the dependency relationship between an as-pect and its corresponding opinion , there probably exist multiple aspectsand different opinion expressions in a sentence. Tojudge the sentiment of a particular aspect, previousstudies (Wang et al., 2016; Tang et al.)

5 , 2016a; Maet al., 2017; Chen et al., 2017; Fan et al., 2018;Huang et al., 2018; Gu et al., 2018) have proposedvarious recurrent neural Networks (RNNs) with at-tention mechanisms to generate aspect-specific sen-tence representations and have achieved appealingresults. However, an inherent defect makes theattention mechanism vulnerable to noise in the sen-tence. Take Figure 1 as an example; for the aspect service , the opinion word reasonable may re-ceive more attention than the opinion word poor .However, the reasonable refers to another as-pect, , price .More recent efforts (Zhang et al., 2019; Sunet al., 2019b; Huang and Carley, 2019; Zhang andQian, 2020; Chen et al.

6 , 2020; Liang et al., 2020;Wang et al., 2020; Tang et al., 2020) have been de-6320voted to Graph Convolutional Networks (GCNs) andgraph attention Networks (GATs) over dependencytrees, which explicitly exploit the syntactic struc-ture of a sentence. Consider the dependency treein Figure 1; the syntactic dependency can establishconnections between the words in a sentence. Forexample, a dependency relation exists between theaspect price and the opinion word reasonable .However, two challenges arise when applying syn-tactic dependency knowledge to the ABSA task: 1)the inaccuracy of the dependency parsing resultsand 2) GCNs over dependency trees do not workwell as expected on datasets that are not sensitiveto syntactic dependency due to the informal expres-sion and complexity of online this paper, we propose a novel architecture,the dual Graph convolution network (DualGCN), asshown in Figure 2, to solve the aforementionedchallenges.

7 For the first challenge, we use theprobability matrix of all dependency arcs from adependency parser to build a syntax-based graphconvolutional network (SynGCN). The idea behindthis approach is that the probability matrix rep-resenting dependencies between words containsrich syntactic information compared with the finaldiscrete output of a dependency parser. For thesecond, we construct a semantic correlation-basedgraph Convolutional network (SemGCN) by utiliz-ing a self-attention mechanism. The idea behindthis approach is that the attention matrix shapedby self-attending, also viewed as an edge-weighteddirected Graph , can represent semantic correlationsbetween words.

8 Moreover, motivated by the workof DGEDT (Tang et al., 2020), we utilize a BiAffinemodule to bridge relevant information between theSynGCN and SemGCN , we design two regularizers to en-hance our DualGCN model. We observe that thesemantically related terms of each word should notoverlap. Therefore, we encourage the attentionprobability distributions over words to be orthog-onal. To this end, we incorporate an orthogonalregularizer on the attention probability matrix forthe SemGCN module. Moreover, the two represen-tations learned from the SynGCN and SemGCNmodules should contain significantly distinct infor-mation captured by the syntactic dependency andthe semantic correlation.

9 Therefore, we expect thatthe SemGCN module could learn semantic repre-sentations different from syntactic , we propose a differential regularizer betweenthe SynGCN and SemGCN contributions are highlighted as follows: We propose a DualGCN model for the ABSA task. Our DualGCN considers both the syntacticstructure and the semantic correlation within agiven sentence. Specifically, our DualGCN in-tegrates the SynGCN and SemGCN networksthrough a mutual BiAffine module. We propose orthogonal and differential regular-izers. The orthogonal regularizer encouragesthe SemGCN network to learn an orthogonal se-mantic attention matrix, whereas the differentialregularizer encourages the SemGCN network tolearn semantic features distinct from the syntac-tic ones built from the SynGCN network.

10 We conduct extensive experiments on the Se-mEval 2014 and Twitter datasets. The experi-mental results demonstrate the effectiveness ofour DualGCN model. Additionally, the sourcecode and preprocessed datasets used in our workare provided on Related WorkTraditional sentiment analysis tasks are sentence-level or document-level contrast,ABSA is an entity-level oriented and a more fine-grained task for sentiment analysis . Earlier meth-ods (Titov and McDonald, 2008; Jiang et al., 2011;Kiritchenko et al., 2014; Vo and Zhang, 2015) areusually based on handcrafted features and fail tomodel the dependency between the given aspectand its , various attention-based neural net-works have been proposed to implicitly model thesemantic relation of an aspect and its context to cap-ture the opinion expression component (Wang et al.)