Abstract:Learning effective representations for Continuous-Time Dynamic Graphs (CTDGs) has garnered significant research interest, largely due to its powerful capabilities in modeling complex interactions between nodes. A fundamental and crucial requirement for representation learning in CTDGs is the appropriate estimation and preservation of proximity. However, due to the sparse and evolving characteristics of CTDGs, the spatial-temporal properties inherent in high-order proximity remain largely unexplored. Despite its importance, this property presents significant challenges due to the computationally intensive nature of personalized interaction intensity estimation and the dynamic attributes of CTDGs. To this end, we propose a novel Correlated Spatial-Temporal Positional encoding that incorporates a parameter-free personalized interaction intensity estimation under the weak assumption of the Poisson Point Process. Building on this, we introduce the Dynamic Graph Transformer with \Correlated Spatial-Temporal Positional Encoding (CorDGT), which efficiently retains the evolving spatial-temporal high-order proximity for effective node representation learning in CTDGs. Extensive experiments on seven small and two large-scale datasets demonstrate the superior performance and scalability of the proposed CorDGT.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of effective representation learning in Continuous - Time Dynamic Graphs (CTDGs). Specifically, the author focuses on how to accurately estimate and preserve the **spatial - temporal high - order proximity** between nodes in CTDGs. #### Main problem background 1. **Sparsity and evolution characteristics of CTDGs**: - In CTDGs, the interactions between nodes are sparse and constantly changing, which makes it difficult to directly apply the methods of traditional static graphs or discrete - time dynamic graphs. - High - order proximity refers to the closeness between the target node and the auxiliary nodes, which reflects the spatial - temporal duality. For example, when predicting the interaction probability between users \(u\) and \(v\) at time \(t = 11\) in a social network, it is necessary to consider their common neighbors, their interaction frequencies and times. 2. **Limitations of existing methods**: - Most existing methods aggregate the neighbor information of the target node independently without considering the dependency relationships between the target nodes. - Some methods encode the positions of auxiliary nodes through random walk paths, but fail to solve the problem of the spatial - temporal duality in high - order proximity. - Calculating high - order proximity requires estimating the personalized interaction intensity between the target node pairs and the auxiliary nodes, which is a computationally intensive task and usually requires pre - training node embeddings or neural networks, resulting in high computational costs. #### Proposed solutions To solve the above problems, the author proposes a new model named **CorDGT (Dynamic Graph Transformer with Correlated Spatial - Temporal Positional Encoding)**. The main contributions of this model include: 1. **Proposing a new spatial - temporal position encoding (STPE - C)**: - By introducing a non - parametric interaction intensity estimation method based on the Poisson point process assumption, the high computational cost of pre - training is avoided. - STPE - C can effectively encode the evolving spatial - temporal high - order proximity in CTDGs, thereby better preserving the comprehensive proximity between nodes. 2. **Designing an efficient dynamic graph Transformer architecture**: - This architecture can adaptively preserve the comprehensive proximity in CTDGs, thereby achieving effective node representation learning. - Through experiments, CorDGT has excellent performance in both link prediction and node classification tasks, and shows a superior efficiency - effectiveness trade - off on large - scale datasets. In conclusion, this paper solves the challenges of high - order proximity modeling in CTDGs by introducing new spatio - temporal encoding and Transformer architecture, significantly improving the effectiveness and efficiency of dynamic graph representation learning.

Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding

Spatial-temporal Graph Convolutional Networks with Diversified Transformation for Dynamic Graph Representation Learning

EGCT: Enhanced Graph Convolutional Transformer for 3D Point Cloud Representation Learning

DTFormer: A Transformer-Based Method for Discrete-Time Dynamic Graph Representation Learning

Tensor Graph Convolutional Network for Dynamic Graph Representation Learning

Continuous-Time Dynamic Graph Learning Via Neural Interaction Processes.

K-Core based Temporal Graph Convolutional Network for Dynamic Graphs

TPGNN: Learning High-order Information in Dynamic Graphs via Temporal Propagation

Dynamic Graph Representation Learning via Edge Temporal States Modeling and Structure-reinforced Transformer

DyG2Vec: Efficient Representation Learning for Dynamic Graphs

Scalable and Effective Temporal Graph Representation Learning With Hyperbolic Geometry

Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective

Long Range Propagation on Continuous-Time Dynamic Graphs

Temporal Graph Network for continuous-time dynamic event sequence

Deformable Graph Transformer

Dyngraphgan: Dynamic Graph Embedding Via Generative Adversarial Networks

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

DECRL: A Deep Evolutionary Clustering Jointed Temporal Knowledge Graph Representation Learning Approach

Time-Aware Dynamic Graph Embedding for Asynchronous Structural Evolution

A Simple But Powerful Graph Encoder for Temporal Knowledge Graph Completion

Learning Spatial-Temporal Feature with Graph Product