Knowledge-Enhanced Hierarchical Graph Transformer …

1 Knowledge-Enhanced Hierarchical Graph Transformer Networkfor Multi-Behavior RecommendationLianghao Xia1, Chao Huang2 , Yong Xu1,3,4, Peng Dai2, Xiyue Zhang1 Hongsheng Yang2, Jian Pei5, Liefeng Bo2 South China University of Technology1, China, JD Finance America Corporation2, USAC ommunication and Computer Network Laboratory of Guangdong3, ChinaPeng Cheng Laboratory, Shenzhen, China, Simon Fraser University5, user and item embedding learning is crucial formodern recommender systems. However, most existing rec-ommendation techniques have thus far focused on model-ing users preferences over singular type of user-item inter-actions. Many practical recommendation scenarios involvemulti-typed user interactive behaviors ( , page view, add-to-favorite and purchase), which presents unique challengesthat cannot be handled by current recommendation particular: i) complex inter-dependencies across differenttypes of user behaviors; ii) the incorporation of knowledge-aware item relations into the multi-behavior recommen-dation framework; iii) dynamic characteristics of multi-typed user-item interactions.

2 To tackle these challenges, thiswork proposes aKnowledge-EnhancedHierarchicalGraphTran sformer Network (KHGT), to investigate multi-typed in-teractive patterns between users and items in recommendersystems. Specifically, KHGT is build upon a Graph -structuredneural architecture to i) capture type-specific behavior se-mantics; ii) explicitly discriminate which types of user-iteminteractions are more important in assisting the forecastingtask on the target behavior. Additionally, we further inte-grate the multi-modal Graph attention layer with temporal en-coding strategy, to empower the learned embeddings be re-flective of both dedicated multiplex user-item and item-itemcollaborative relations, as well as the underlying interactiondynamics.

3 Extensive experiments conducted on three real-world datasets show that KHGT consistently outperformsmany state-of-the-art recommendation methods across vari-ous evaluation settings. Our implementation is available systems have been widely deployed in manyInternet services ( , e-commerce, online review and ad-vertising systems) to alleviate information overload and de-liver the most relevant to users (Liu et al. 2020; Huang etal. 2019a). In the recommendation scenario with the focuson implicit feedback, Collaborative Filtering (CF) becomesone of most popular paradigm which factorizes user-iteminteractions into latent representation and predicts user s Corresponding author: Chao HuangCopyrightc 2020, Association for the Advancement of ArtificialIntelligence ( ).

4 All rights based on the projected low-dimensional embed-dings (Chen et al. 2020).In recent years, many deep neural network techniqueshave been developed to enhance collaborative filtering ar-chitecture for non-linear feature interactions. Specifically,early studies, like NCF (He et al. 2017) and DMF (Xue etal. 2017) utilizes the Multi-layer Perceptron to handle thehigh-level non-linearities. Furthermore, autoencoder-basedmethods are designed for mapping high-dimensional sparseuser-item interactions into low-dimensional dense latent rep-resentations (Sedhain et al. 2015; Wu, DuBois, and others2016). Later works investigate the use of Graph neural net-work to exploit the Graph -structured high-order user-item re-lations, and perform neighborhood-based feature aggrega-tion (Zhang et al.)

5 2019; Wang et al. 2019c).Although these methods have shown promising results,a deficiency is that they only model singular type of user-item interactions, which makes them insufficient to distillthe complex collaborative signals from the multi-typed be-haviors of users (Jin et al. 2020). In particular, there typicallyexist multiple relations between user and item that exhibitvarious behavior semantics in many real-world recommen-dation scenarios, which are particularly helpful in learningusers preferences on the target type of behavior (Guo et ). For example, in online retail platforms, users pageview and add-to-favorite activities over different items, canserve as the auxiliary knowledge for assisting the forecastingtask of customer purchase intent (target behavior).

6 There-fore, it is crucial to take such inter-type behavioral influencesinto consideration to more accurately infer user are several key technical challenges that remainto be solved to realize the multi-behavior , how to distill the user-specific collaborative signalsfrom the multiplex user-item interactive behaviors, is a sig-nificant challenge to tackle. In practice, type-specific behav-ioral patterns interweave with each other in a complex man-ner and vary by users (Gao et al. 2019b), like the comple-mentary correlations between the add-to-cart and purchasebehaviors, or users negative reviews are mutually exclusivewith their positive feedback over the same item.

7 Without theexplicitly encoding of such heterogeneous relationships be-tween user and item, models may suffer from the inabilityPRELIMINARY VERSION: DO NOT CITE The AAAI Digital Library will contain the published version some time after the conferenceof capturing the complicated inherent cross-type behaviordependencies in the Hierarchical , another corechallenge lies in the incorporation of knowledge-aware itemsemantic relatedness into the encoding function of multi-behavioral patterns. The knowledge-aware side informationoften contains much fruitful facts and contextual connec-tions about items (Wang et al. 2019a). It is desirable to rigor-ously design a joint embedding paradigm over the user-itemand item-item relations in our multi-behavior , a time-aware model is needed to better handlethe dynamic structural dependency of user-item are a handful of recent models that attempt to inte-grate multi-behavioral interactive patterns for making rec-ommendations (Jin et al.)

8 2020; Gao et al. 2019b). How-ever, these works intend to consider multi-typed interactionsin a relatively independent and local manner ( , singu-lar dimensional cascading correlations), and can hardly cap-ture the high-order multiplex relationships across users anditems. Additionally, how to account for the side knowledgefrom items as well as user-item interaction dynamics, is lessexplored in those multi-behavior recommender light of these differences and challenges, we presenta general framework Knowledge-EnhancedHierarchicalGraphTrans former Network (KHGT), for multi-behaviorrecommendation. Particularly, at the first stage, we developa multi-behavior Graph Transformer network which performsrecursive embedding propagation, to capture high-order ofbehavior heterogeneity across users and items in an atten-tive aggregation schema.

9 As a result, user-item and item-item relationships of different types are enabled to main-tain their specific representation spaces. To handle behaviordynamics, we inject the time-aware context into the graphtransformer framework through a temporal encoding strat-egy. In addition, to encode the inter-dependencies betweentype-specific behavior representations, a multi-behavior mu-tual attention encoder is proposed to learn dependent struc-tures of different types of behaviors in a pairwise , a gated aggregation layer is introduced to discrim-inate the contribution of type-specific relation embeddingsfor making final contributions of this paper are highlighted as follows: We propose a framework KHGT, which explicitlyachieves high-order relation learning in the knowledge-aware multi-behavior collaborative Graph under the hier-archically structured Graph Transformer network.

10 To jointly integrate user- and item-wise collaborativesimilarities under the multi-behavior modeling paradigmof KHGT: i) the first-stage Graph -structured transformermodule captures the type-specific user-item interactivepatterns in a time-aware environment; ii) the second-stageattentive fusion network encodes the cross-type behav-ior Hierarchical dependencies and discriminates the type-specific contribution, in forecasting the target behaviors. We apply the proposed KHGT method to three real-worlddatasets of movie, venue and product show that our model achieves significantgains over 15 state-of-the-art baselines from various , model interpretation ability is also investi-gated with case studies of qualitative begin with the introduction of key notations and con-sider a typical recommendation scenario withIusers (U={u1.}