Abstract:Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various applications. Such temporal graphs exhibit heterogeneous transient dynamics, varying time intervals, and highly evolving node features throughout their evolution. Hence, incorporating long-range dependencies from the historical graph context plays a crucial role in accurately learning their temporal dynamics. In this paper, we develop a graph embedding model with uncertainty quantification, TransformerG2G, by exploiting the advanced transformer encoder to first learn intermediate node representations from its current state ($t$) and previous context (over timestamps [$t-1, t-l$], $l$ is the length of context). Moreover, we employ two projection layers to generate lower-dimensional multivariate Gaussian distributions as each node's latent embedding at timestamp $t$. We consider diverse benchmarks with varying levels of ``novelty" as measured by the TEA (Temporal Edge Appearance) plots. Our experiments demonstrate that the proposed TransformerG2G model outperforms conventional multi-step methods and our prior work (DynG2G) in terms of both link prediction accuracy and computational efficiency, especially for high degree of novelty. Furthermore, the learned time-dependent attention weights across multiple graph snapshots reveal the development of an automatic adaptive time stepping enabled by the transformer. Importantly, by examining the attention weights, we can uncover temporal dependencies, identify influential elements, and gain insights into the complex interactions within the graph structure. For example, we identified a strong correlation between attention weights and node degree at the various stages of the graph topology evolution.

Generating Biomedical Hypothesis with Spatiotemporal Transformers

Contrasting Multi-Source Temporal Knowledge Graphs for Biomedical Hypothesis Generation

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

GTHP: a novel graph transformer Hawkes process for spatiotemporal event prediction

Relation-aware Heterogeneous Graph Transformer Based Drug Repurposing

Enhancing Time Series Forecasting: A Hierarchical Transformer with Probabilistic Decomposition Representation

A bayesian multivariate mixture model for high throughput spatial transcriptomics

Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs

AGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approach

Spatial transcriptomics prediction from histology jointly through Transformer and graph neural networks

Supervised Learning Based Hypothesis Generation from Biomedical Literature.

Spatially Resolved Gene Expression Prediction from Histology via Multi-view Graph Contrastive Learning with HSIC-bottleneck Regularization

Spatiotemporal Fusion Transformer for large-scale traffic forecasting

Time-aware Heterogeneous Graph Transformer with Adaptive Attention Merging for Health Event Prediction

Temporal Link Prediction Via Auxiliary Graph Transformer

Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space

Tigc-Net: Transformer-Improved Graph Convolution Network for Spatio-Temporal Prediction

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

Hybrid-TE: Hybrid Translation-Based Temporal Knowledge Graph Embedding

Hierarchical graph transformer with contrastive learning for gene regulatory network inference

Hyper-STTN: Social Group-aware Spatial-Temporal Transformer Network for Human Trajectory Prediction with Hypergraph Reasoning