Abstract:Dynamic graph representation learning has emerged as a crucial research area, driven by the growing need for analyzing time-evolving graph data in real-world applications. While recent approaches leveraging recurrent neural networks (RNNs) and graph neural networks (GNNs) have shown promise, they often fail to adequately capture the impact of temporal edge states on inter-node relationships, consequently overlooking the dynamic changes in node features induced by these evolving relationships. Furthermore, these methods suffer from GNNs' inherent over-smoothing problem, which hinders the extraction of global structural features. To address these challenges, we introduce the Recurrent Structure-reinforced Graph Transformer (RSGT), a novel framework for dynamic graph representation learning. It first designs a heuristic method to explicitly model edge temporal states by employing different edge types and weights based on the differences between consecutive snapshots, thereby integrating varying edge temporal states into the graph's topological structure. We then propose a structure-reinforced graph transformer that captures temporal node representations encoding both graph topology and evolving dynamics through a recurrent learning paradigm, enabling the extraction of both local and global structural features. Comprehensive experiments on four real-world datasets demonstrate RSGT's superior performance in discrete dynamic graph representation learning, consistently outperforming existing methods in dynamic link prediction tasks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in dynamic graph representation learning, existing methods are often unable to fully capture the influence of the temporal state of edges on the relationships between nodes, thus ignoring the dynamic changes of node features caused by these evolving relationships. In addition, existing methods are also affected by the inherent over - smoothing problem of graph neural networks (GNNs), which hinders the extraction of global structural features. Specifically: 1. **Influence of edge temporal state**: Current methods usually do not explicitly model the temporal state of edges (such as the duration, appearance or disappearance of edges), and these states significantly affect node characteristics. Many methods fail to explicitly model these dynamic changes. 2. **Over - smoothing problem**: Although recent research has begun to explore the use of Transformer architectures to solve the over - smoothing problem in GNNs, there are still challenges in effectively integrating graph structural information with complex temporal dependencies. 3. **Utilization of historical node information**: Many existing methods have difficulty in using historical node information inductively when dealing with evolving networks, which limits their application in dynamic networks. To address these challenges, the paper proposes a new framework - Recurrent Structure - reinforced Graph Transformer (RSGT). RSGT solves the above problems in the following ways: - **Edge temporal state modeling**: RSGT designs a heuristic method to explicitly model the temporal state of edges by using different edge types and weights, and integrates the changing edge temporal states into the topological structure of the graph based on the differences between consecutive snapshots. - **Structure - enhanced graph Transformer**: RSGT introduces a structure - enhanced graph Transformer, which captures the temporal node representations encoding the graph topology and evolution dynamics through a recursive learning paradigm, thereby being able to extract local and global structural features. - **Recursive learning paradigm**: RSGT regards dynamic graph representation learning as a series of recursive learning tasks, starting from the initial node features, and continuously updates these representations according to the evolution of the graph structure and edge states, thereby effectively capturing the long - term dependencies in dynamic graphs. Through these innovations, RSGT performs excellently in the dynamic link prediction tasks on four real - world datasets, significantly outperforming existing methods.

Dynamic Graph Representation Learning via Edge Temporal States Modeling and Structure-reinforced Transformer

Dynamic Representation Learning Via Recurrent Graph Neural Networks

DTFormer: A Transformer-Based Method for Discrete-Time Dynamic Graph Representation Learning

RDGSL: Dynamic Graph Representation Learning with Structure Learning

Temporal Graph Transformer for Dynamic Network

Learning Continuous Dynamic Network Representation with Transformer-Based Temporal Graph Neural Network.

Efficient Dynamic Graph Representation Learning at Scale

Time-Aware Dynamic Graph Embedding for Asynchronous Structural Evolution

TemporalGAT: Attention-Based Dynamic Graph Representation Learning

A Hybrid Continuous-Time Dynamic Graph Representation Learning Model by Exploring Both Temporal and Repetitive Information

Spatial-temporal Graph Convolutional Networks with Diversified Transformation for Dynamic Graph Representation Learning

A recurrent graph neural network for inductive representation learning on dynamic graphs

Capturing Temporal Node Evolution Via Self-supervised Learning: A New Perspective on Dynamic Graph Learning.

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

Temporal Resonant Graph Network for Representation Learning on Dynamic Graphs

Dynamic Graph Representation Based on Temporal and Contextual Contrasting.

Scaling Up Dynamic Graph Representation Learning Via Spiking Neural Networks

Dynamic Graph Representation Learning Via Coupling-Process Model.

Dyngraphgan: Dynamic Graph Embedding Via Generative Adversarial Networks

Tensor Graph Convolutional Network for Dynamic Graph Representation Learning

Temporal Link Prediction Via Auxiliary Graph Transformer