Neural Graph Collaborative Filtering - USTC

1 Neural Graph Collaborative FilteringXiang WangNational University of He University of Science and Technologyof WangHefei University of FengNational University of ChuaNational University of vector representations ( ) of users anditems lies at the core of modern recommender systems. Rangingfrom early matrix factorization to recently emerged deep learningbased methods, existing efforts typically obtain a user s (or anitem s) embedding by mapping from pre-existing features thatdescribe the user (or the item), such as ID and attributes. Weargue that an inherent drawback of such methods is that, thecollaborative signal, which is latent in user-item interactions,is not encoded in the embedding process. As such, the resultantembeddings may not be sufficient to capture the collaborativefiltering this work, we propose to integrate the user-item interactions more specifically the bipartite Graph structure into the embeddingprocess.

2 We develop a new recommendation frameworkNeuralGraph Collaborative Filtering (NGCF), which exploits the user-item Graph structure by propagating embeddings on it. This leadsto the expressive modeling ofhigh-order connectivityin user-item Graph , effectively injecting the Collaborative signal into theembedding process in an explicit manner. We conduct extensiveexperiments on three public benchmarks, demonstrating significantimprovements over several state-of-the-art models like HOP-Rec [38] and Collaborative Memory Network [5]. Further analysisverifies the importance of embedding propagation for learningbetter user and item representations, justifying the rationality andeffectiveness of CONCEPTS Information systems Recommender Filtering , Recommendation, High-order Connectivity,Embedding Propagation, Graph Neural Network Xiangnan He is the corresponding to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page.

3 Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from 19, July 21 25, 2019, Paris, France 2019 Association for Computing ISBN 978-1-4503-6172-9/19/07.. $ Reference Format:Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Graph Collaborative Filtering . InProceedings of the 42nd InternationalACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR 19), July 21 25, 2019, Paris, , New York, NY, USA,10 pages. INTRODUCTIONP ersonalized recommendation is ubiquitous, having been appliedto many online services such as E-commerce, advertising, andsocial media.

4 At its core is estimating how likely a user willadopt an item based on the historical interactions like purchasesand clicks. Collaborative Filtering (CF) addresses it by assumingthat behaviorally similar users would exhibit similar preferenceon items. To implement the assumption, a common paradigmis to parameterize users and items for reconstructing historicalinteractions, and predict user preference based on the parameters [1,14].Generally speaking, there are two key components in learnableCF models 1)embedding, which transforms users and itemsto vectorized representations, and 2)interaction modeling, whichreconstructs historical interactions based on the embeddings. Forexample, matrix factorization (MF) directly embeds user/item ID asan vector and models user-item interaction with inner product [20]; Collaborative deep learning extends the MF embedding functionby integrating the deep representations learned from rich sideinformation of items [29]; Neural Collaborative Filtering modelsreplace the MF interaction function of inner product with nonlinearneural networks [14]; and translation-based CF models instead useEuclidean distance metric as the interaction function [27], their effectiveness, we argue that these methods are notsufficient to yield satisfactory embeddings for CF.

5 The key reason isthat the embedding function lacks an explicit encoding of the crucialcollaborative signal, which is latent in user-item interactions toreveal the behavioral similarity between users (or items). To bemore specific, most existing methods build the embedding functionwith the descriptive features only ( ,ID and attributes), withoutconsidering theuser-item interactions which are only used todefine the objective function for model training [26,27]. As a result,when the embeddings are insufficient in capturing CF, the methodshave to rely on the interaction function to make up for the deficiencyof suboptimal embeddings [14].While intuitively useful to integrate user-item interactionsinto the embedding function, it is non-trivial to do it well. InFigure 1: An illustration of the user-item interaction graphand the high-order connectivity.

6 The nodeu1is the targetuser to provide recommendations , the scale of interactions can easily reach millionsor even larger in real applications, making it difficult to distillthe desired Collaborative signal. In this work, we tackle thechallenge by exploiting thehigh-order connectivityfrom user-item interactions, a natural way that encodes Collaborative signalin the interaction Graph Example. Figure 1 illustrates the concept of high-orderconnectivity. The user of interest for recommendation isu1, labeledwith the double circle in the left subfigure of user-item interactiongraph. The right subfigure shows the tree structure that is expandedfromu1. The high-order connectivity denotes the path that reachesu1from any node with the path lengthllarger than 1. Such high-order connectivity contains rich semantics that carry collaborativesignal.

7 For example, the pathu1 i2 u2indicates the behaviorsimilarity betweenu1andu2, as both users have interacted withi2; the longer pathu1 i2 u2 i4suggests thatu1is likely toadopti4, since her similar useru2has consumedi4before. Moreover,from the holistic view ofl= 3, itemi4is more likely to be of interesttou1than itemi5, since there are two paths connecting <i4,u1>,while only one path connects <i5,u1>.Present Work. We propose to model the high-order connectivityinformation in the embedding function. Instead of expandingthe interaction Graph as a tree which is complex to implement,we design a Neural network method to propagate embeddingsrecursively on the Graph . This is inspired by the recentdevelopments of Graph Neural networks [8,30,36], which can beseen as constructing information flows in the embedding , we devise anembedding propagationlayer, whichrefines a user s (or an item s) embedding by aggregating theembeddings of the interacted items (or users).

8 By stacking multipleembedding propagation layers, we can enforce the embeddingsto capture the Collaborative signal in high-order Figure 1 as an example, stacking two layers captures thebehavior similarity ofu1 i2 u2, stacking three layers capturesthe potential recommendations ofu1 i2 u2 i4, and thestrength of the information flow (which is estimated by the trainableweights between layers) determines the recommendation priorityofi4andi5. We conduct extensive experiments on three publicbenchmarks to verify the rationality and effectiveness of ourNeuralGraph Collaborative Filtering (NGCF) , it is worth mentioning that although the high-orderconnectivity information has been considered in a very recentmethod named HOP-Rec [38], it is only exploited to enrichthe training data.

9 Specifically, the prediction model of HOP-Rec remains to be MF, while it is trained by optimizing a lossthat is augmented with high-order connectivities. Distinct fromHOP-Rec, we contribute a new technique to integrate high-orderconnectivities into the prediction model, which empirically yieldsbetter embeddings than HOP-Rec for summarize, this work makes the following main contributions: We highlight the critical importance of explicitly exploiting thecollaborative signal in the embedding function of model-basedCF methods. We propose NGCF, a new recommendation framework based ongraph Neural network, which explicitly encodes the collaborativesignal in the form of high-order connectivities by performingembedding propagation. We conduct empirical studies on three million-size results demonstrate the state-of-the-art performance ofNGCF and its effectiveness in improving the embedding qualitywith Neural embedding METHODOLOGYWe now present the proposed NGCF model, the architecture ofwhich is illustrated in Figure 2.

10 There are three components in theframework: (1) an embedding layer that offers and initializationof user embeddings and item embeddings; (2) multiple embeddingpropagation layers that refine the embeddings by injecting high-order connectivity relations; and (3) the prediction layer thataggregates the refined embeddings from different propagationlayers and outputs the affinity score of a user-item pair. Finally,we discuss the time complexity of NGCF and the connections withexisting Embedding LayerFollowing mainstream recommender models [1,14,26], we describea useru(an itemi) with an embedding vectoreu Rd(ei Rd),whereddenotes the embedding size. This can be seen as building aparameter matrix as an embedding look-up table:E= [eu1, ,euN| {z }users embeddings,ei1, ,eiM| {z }item embeddings].