Abstract:Graph Neural Networks (GNNs) have emerged as promising solutions for collaborative filtering (CF) through the modeling of user-item interaction graphs. The nucleus of existing GNN-based recommender systems involves recursive message passing along user-item interaction edges to refine encoded embeddings. Despite their demonstrated effectiveness, current GNN-based methods encounter challenges of limited receptive fields and the presence of noisy "interest-irrelevant" connections. In contrast, Transformer-based methods excel in aggregating information adaptively and globally. Nevertheless, their application to large-scale interaction graphs is hindered by inherent complexities and challenges in capturing intricate, entangled structural information. In this paper, we propose TransGNN, a novel model that integrates Transformer and GNN layers in an alternating fashion to mutually enhance their capabilities. Specifically, TransGNN leverages Transformer layers to broaden the receptive field and disentangle information aggregation from edges, which aggregates information from more relevant nodes, thereby enhancing the message passing of GNNs. Additionally, to capture graph structure information effectively, positional encoding is meticulously designed and integrated into GNN layers to encode such structural knowledge into node attributes, thus enhancing the Transformer's performance on graphs. Efficiency considerations are also alleviated by proposing the sampling of the most relevant nodes for the Transformer, along with two efficient sample update strategies to reduce complexity. Furthermore, theoretical analysis demonstrates that TransGNN offers increased expressiveness compared to GNNs, with only a marginal increase in linear complexity. Extensive experiments on five public datasets validate the effectiveness and efficiency of TransGNN.

Diffuse and Smooth: Beyond Truncated Receptive Field for Scalable and Adaptive Graph Representation Learning.

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Diffusion Improves Graph Learning

Adaptive Graph Diffusion Networks

Scalable Graph Compressed Convolutions

Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals

Deep Graph Neural Networks via Flexible Subgraph Aggregation

Diffusing Graph Attention

Enhanced Scalable Graph Neural Network via Knowledge Distillation

Graph Neural Diffusion Networks for Semi-supervised Learning

A Unified View on Graph Neural Networks as Graph Signal Denoising

Advective Diffusion Transformers for Topological Generalization in Graph Learning

Multi-Scale Graph Convolutional Network with Spectral Graph Wavelet Frame.

TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systems

Spectral Graph Convolutional Networks WithLifting-based Adaptive Graph Wavelets

TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning

Adaptive Sampling Towards Fast Graph Representation Learning

Towards Scale-Invariant Graph-related Problem Solving by Iterative Homogeneous Graph Neural Networks

Decoupling the Depth and Scope of Graph Neural Networks

Graph Dilated Network with Rejection Mechanism

AdaptiveGCN: Efficient GCN Through Adaptively Sparsifying Graphs