Abstract:Graph Neural Networks (GNNs) have emerged as promising solutions for collaborative filtering (CF) through the modeling of user-item interaction graphs. The nucleus of existing GNN-based recommender systems involves recursive message passing along user-item interaction edges to refine encoded embeddings. Despite their demonstrated effectiveness, current GNN-based methods encounter challenges of limited receptive fields and the presence of noisy "interest-irrelevant" connections. In contrast, Transformer-based methods excel in aggregating information adaptively and globally. Nevertheless, their application to large-scale interaction graphs is hindered by inherent complexities and challenges in capturing intricate, entangled structural information. In this paper, we propose TransGNN, a novel model that integrates Transformer and GNN layers in an alternating fashion to mutually enhance their capabilities. Specifically, TransGNN leverages Transformer layers to broaden the receptive field and disentangle information aggregation from edges, which aggregates information from more relevant nodes, thereby enhancing the message passing of GNNs. Additionally, to capture graph structure information effectively, positional encoding is meticulously designed and integrated into GNN layers to encode such structural knowledge into node attributes, thus enhancing the Transformer's performance on graphs. Efficiency considerations are also alleviated by proposing the sampling of the most relevant nodes for the Transformer, along with two efficient sample update strategies to reduce complexity. Furthermore, theoretical analysis demonstrates that TransGNN offers increased expressiveness compared to GNNs, with only a marginal increase in linear complexity. Extensive experiments on five public datasets validate the effectiveness and efficiency of TransGNN.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address several core challenges in graph neural network (GNN)-based methods for recommendation systems and proposes a new framework, TransGNN, which combines Transformer with GNN to enhance the performance of recommendation systems. #### Main Issues 1. **Limited Receptive Field and Noise Connection Issues**: - Existing GNN methods are limited by the local receptive field in the user-item interaction graph during information propagation, making them susceptible to noise interference, which leads to inaccurate information transmission. - Users may not be interested in items ranked lower in the recommendation list and tend to browse items at the top of the list, regardless of their actual relevance. - Users may interact with items they are not genuinely interested in because the recommendation system over-recommends popular items, forming "interest-irrelevant connections" in the user-item interaction graph. 2. **Over-Smoothing Problem**: - Deep GNN models tend to suffer from over-smoothing during training, where node representations become too similar, making it difficult to distinguish between different nodes. - In real-world applications, item sequences typically exceed a length of 3, but existing GNN models can only capture relationships up to 3 hops, failing to capture longer sequence patterns. 3. **Limitations of Transformer**: - Although Transformer can globally adaptively aggregate information, it falls short in utilizing graph structural information because it does not rely on edges for information aggregation, thus underestimating critical historical interactions. #### Solutions To address the above challenges, the paper proposes the TransGNN model, which achieves the following: 1. **Attention Sampling Module**: - Samples the most relevant nodes for each central node, considering semantic similarity and graph structural information, reducing computational complexity and filtering out noise information. 2. **Position Encoding Module**: - Introduces three types of position encodings: shortest path position encoding, degree position encoding, and PageRank position encoding, embedding structural information into node representations to enhance the Transformer's understanding of graph topology. 3. **TransGNN Module**: - Alternates between Transformer layers and GNN layers to mutually enhance performance. The Transformer layer extends the GNN's receptive field, while the GNN layer helps the Transformer better understand graph structural information. - Proposes two efficient methods for updating attention samples, reducing computational complexity. Through these methods, TransGNN not only addresses the limitations of existing GNN models but also improves the expressiveness and efficiency of recommendation systems.

TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systems

Can Transformer and GNN Help Each Other?

Graph Transformer Networks Based Text Representation.

Position-aware Graph Transformer for Recommendation

TRANSGNN: Towards Knowledge Enhanced Top-K Recommendation via Transformed Heterogeneous Graph Neural Network.

Graph Trend Filtering Networks for Recommendations

Collaborative Sequential Recommendations via Multi-View GNN-Transformers

Graph Trend Filtering Networks for Recommendation

How Expressive are Graph Neural Networks in Recommendation?

GTC: GNN-Transformer co-contrastive learning for self-supervised heterogeneous graph representation

A Graph Neural Network Framework for Social Recommendations

HybridGNN: Learning Hybrid Representation for Recommendation in Multiplex Heterogeneous Networks

TGCN: Tag Graph Convolutional Network for Tag-Aware Recommendation

TransGNN: A Transductive Graph Neural Network with Graph Dynamic Embedding

Graph-Coupled Time Interval Network for Sequential Recommendation

Graph Transformer: Learning Better Representations for Graph Neural Networks.

HybridGNN: Learning Hybrid Representation in Multiplex Heterogeneous Networks

Sequential Recommendation through Graph Neural Networks and Transformer Encoder with Degree Encoding

TransNets: Learning to Transform for Recommendation

TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning