LightGCN: Simplifying and Powering Graph Convolution ...

1 lightgcn : Simplifying and Powering Graph ConvolutionNetwork for RecommendationXiangnan HeUniversity of Science and Technologyof DengUniversity of Science and Technologyof WangNational University of LiBeijing Kuaishou TechnologyCo., ZhangUniversity of Science and Technologyof Wang Hefei University of Convolution Network (GCN) has become new state-of-the-art for collaborative filtering. Nevertheless, the reasons ofits effectiveness for recommendation are not well work that adapts GCN to recommendation lacks thoroughablation analyses on GCN, which is originally designed for graphclassification tasks and equipped with many neural networkoperations. However, we empirically find that the two mostcommon designs in GCNs feature transformation and nonlinearactivation contribute little to the performance of collaborativefiltering.

2 Even worse, including them adds to the difficulty oftraining and degrades recommendation this work, we aim to simplify the design of GCN tomake it more concise and appropriate for recommendation. Wepropose a new model named lightgcn , including only the mostessential component in GCN neighborhood aggregation forcollaborative filtering. Specifically, lightgcn learns user anditem embeddings by linearly propagating them on the user-iteminteraction Graph , and uses the weighted sum of the embeddingslearned at all layers as the final embedding. Such simple, linear,and neat model is much easier to implement and train, exhibitingsubstantial improvements (about relative improvement onaverage) over Neural Graph Collaborative Filtering (NGCF) astate-of-the-art GCN-based recommender model under exactlythe same experimental setting.

3 Further analyses are providedtowards the rationality of the simple lightgcn from both analyticaland empirical perspectives. Our implementations are available inboth TensorFlow1and PyTorch2. Meng Wang is the corresponding to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from 20, July 25 30, 2020, Virtual Event, China 2020 Copyright held by the owner/author(s).

4 Publication rights licensed to ISBN 978-1-4503-8016-4/20/07.. $ CONCEPTS Information systems Recommender Filtering, Recommendation, Embedding Propagation, Graph Neural NetworkACM Reference Format:Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and MengWang. 2020. lightgcn : Simplifying and Powering Graph ConvolutionNetwork for Recommendation. InProceedings of the 43rd International ACMSIGIR Conference on Research and Development in Information Retrieval(SIGIR 20), July 25 30, 2020, Virtual Event, , New York, NY, USA,10 pages. INTRODUCTIONTo alleviate information overload on the web, recommender systemhas been widely deployed to perform personalized informationfiltering [7,45,46]. The core of recommender system is to predictwhether a user will interact with an item, , click, rate, purchase,among other forms of interactions.

5 As such, collaborative filtering(CF), which focuses on exploiting the past user-item interactions toachieve the prediction, remains to be a fundamental task towardseffective personalized recommendation [10, 19, 28, 39].The most common paradigm for CF is to learn latent features( embedding) to represent a user and an item, and performprediction based on the embedding vectors [6,19]. Matrixfactorization is an early such model, which directly projects thesingle ID of a user to her embedding [26]. Later on, several researchfind that augmenting user ID with the her interaction history asthe input can improve the quality of embedding. For example,SVD++ [25] demonstrates the benefits of user interaction historyin predicting user numerical ratings, and Neural Attentive ItemSimilarity (NAIS) [18] differentiates the importance of items inthe interaction history and shows improvements in predictingitem ranking.

6 In view of user-item interaction Graph , theseimprovements can be seen as coming from using the subgraphstructure of a user more specifically, her one-hop neighbors toimprove the embedding deepen the use of subgraph structure with high-hopneighbors, Wang et al. [39] recently proposes NGCF and achievesstate-of-the-art performance for CF. It takes inspiration from theGraph Convolution Network (GCN) [14,23], following the [ ] 7 Jul 2020propagation rule to refine embeddings: feature transformation,neighborhood aggregation, and nonlinear activation. AlthoughNGCF has shown promising results, we argue that its designsare rather heavy and burdensome many operations are directlyinherited from GCN without justification. As a result, they are notnecessarily useful for the CF task.

7 To be specific, GCN is originallyproposed for node classification on attributed Graph , where eachnode has rich attributes as input features; whereas in user-iteminteraction Graph for CF, each node (user or item) is only describedby a one-hot ID, which has no concrete semantics besides beingan identifier. In such a case, given the ID embedding as the input,performing multiple layers of nonlinear feature transformation which is the key to the success of modern neural networks [16] will bring no benefits, but negatively increases the difficulty formodel validate our thoughts, we perform extensive ablation studieson NGCF. With rigorous controlled experiments (on the same datasplits and evaluation protocol), we draw the conclusion that thetwo operations inherited from GCN feature transformation andnonlinear activation has no contribution on NGCF s more surprising, removing them leads to significant accuracyimprovements.

8 This reflects the issues of adding operations thatare useless for the target task in Graph neural network, which notonly brings no benefits, but rather degrades model by these empirical findings, we present a new modelnamed lightgcn , including the most essential component ofGCN neighborhood aggregation for collaborative , after associating each user (item) with an ID embedding,we propagate the embeddings on the user-item interaction graphto refine them. We then combine the embeddings learned atdifferent propagation layers with a weighted sum to obtain the finalembedding for prediction. The whole model is simple and elegant,which not only is easier to train, but also achieves better empiricalperformance than NGCF and other state-of-the-art methods likeMult-VAE [28].

9 To summarize, this work makes the following main contributions: We empirically show that two common designs in GCN,feature transformation and nonlinear activation, have nopositive effect on the effectiveness of collaborative filtering. We propose lightgcn , which largely simplifies the modeldesign by including only the most essential components inGCN for recommendation. We empirically compare lightgcn with NGCF by followingthe same setting and demonstrate substantial analyses are provided towards the rationality ofLightGCN from both technical and empirical PRELIMINARIESWe first introduce NGCF [39], a representative and state-of-the-artGCN model for recommendation. We then perform ablation studieson NGCF to judge the usefulness of each operation in NGCF. Thenovel contribution of this section is to show that the two commondesigns in GCNs, feature transformation and nonlinear activation,have no positive effect on collaborative 1: Performance of NGCF and its three ndcgrecall NGCF BriefIn the initial step, each user and item is associated with an IDembedding.

10 Lete(0)udenote the ID embedding of useruande(0)idenote the ID embedding of itemi. Then NGCF leverages the user-item interaction Graph to propagate embeddings as:e(k+1)u= W1e(k)u+Xi Nu1p|Nu||Ni|(W1e(k)i+W2(e(k)i e(k)u)) ,e(k+1)i= W1e(k)i+Xu Ni1p|Nu||Ni|(W1e(k)u+W2(e(k)u e(k)i)) ,(1)wheree(k)uande(k)irespectively denote the refined embedding ofuseruand itemiafterklayers propagation, is the nonlinearactivation function,Nudenotes the set of items that are interactedby useru,Nidenotes the set of users that interact with itemi,andW1andW2are trainable weight matrix to perform featuretransformation in each layer. By propagatingLlayers, NGCF obtainsL+ 1embeddings to describe a user (e(0)u,e(1)u,..,e(L)u) and an item(e(0)i,e(1)i,..,e(L)i). It then concatenates theseL+ 1embeddings toobtain the final user embedding and item embedding, using innerproduct to generate the prediction largely follows the standard GCN [23], including the useof nonlinear activation function ( )and feature transformationmatricesW1andW2.