Abstract:Graph Neural Networks (GNNs) have emerged as promising solutions for collaborative filtering (CF) through the modeling of user-item interaction graphs. The nucleus of existing GNN-based recommender systems involves recursive message passing along user-item interaction edges to refine encoded embeddings. Despite their demonstrated effectiveness, current GNN-based methods encounter challenges of limited receptive fields and the presence of noisy "interest-irrelevant" connections. In contrast, Transformer-based methods excel in aggregating information adaptively and globally. Nevertheless, their application to large-scale interaction graphs is hindered by inherent complexities and challenges in capturing intricate, entangled structural information. In this paper, we propose TransGNN, a novel model that integrates Transformer and GNN layers in an alternating fashion to mutually enhance their capabilities. Specifically, TransGNN leverages Transformer layers to broaden the receptive field and disentangle information aggregation from edges, which aggregates information from more relevant nodes, thereby enhancing the message passing of GNNs. Additionally, to capture graph structure information effectively, positional encoding is meticulously designed and integrated into GNN layers to encode such structural knowledge into node attributes, thus enhancing the Transformer's performance on graphs. Efficiency considerations are also alleviated by proposing the sampling of the most relevant nodes for the Transformer, along with two efficient sample update strategies to reduce complexity. Furthermore, theoretical analysis demonstrates that TransGNN offers increased expressiveness compared to GNNs, with only a marginal increase in linear complexity. Extensive experiments on five public datasets validate the effectiveness and efficiency of TransGNN.

Graph Transformer Networks Based Text Representation.

GTA: Graph Transformer Adapter

Graph Transformer: Learning Better Representations for Graph Neural Networks.

TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning

Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks?

Hypergraph Transformer Neural Networks

Rethinking Graph Transformer Architecture Design for Node Classification

TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systems

GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph

Text Level Graph Neural Network for Text Classification

Graph Fusion Network for Text Classification

Graph Transformer for Graph-to-Sequence Learning

Temporal Graph Transformer for Dynamic Network

Can Transformer and GNN Help Each Other?

Recursive Graphical Neural Networks for Text Classification

Tensor-view Topological Graph Neural Network

Text classification optimization algorithm based on graph neural network

GTLP: A Graph Transformer Based Link Prediction Framework with Unified Feature Pre-Processing

Graph topology enhancement for text classification

Sparse Graph Transformer with Contrastive Learning