Abstract:Graph Neural Networks (GNNs) have emerged as fundamental tools for a wide range of prediction tasks on graph-structured data. Recent studies have drawn analogies between GNN feature propagation and diffusion processes, which can be interpreted as dynamical systems. In this paper, we delve deeper into this perspective by connecting the dynamics in GNNs to modern Koopman theory and its numerical method, Dynamic Mode Decomposition (DMD). We illustrate how DMD can estimate a low-rank, finite-dimensional linear operator based on multiple states of the system, effectively approximating potential nonlinear interactions between nodes in the graph. This approach allows us to capture complex dynamics within the graph accurately and efficiently. We theoretically establish a connection between the DMD-estimated operator and the original dynamic operator between system states. Building upon this foundation, we introduce a family of DMD-GNN models that effectively leverage the low-rank eigenfunctions provided by the DMD algorithm. We further discuss the potential of enhancing our approach by incorporating domain-specific constraints such as symmetry into the DMD computation, allowing the corresponding GNN models to respect known physical properties of the underlying system. Our work paves the path for applying advanced dynamical system analysis tools via GNNs. We validate our approach through extensive experiments on various learning tasks, including directed graphs, large-scale graphs, long-range interactions, and spatial-temporal graphs. We also empirically verify that our proposed models can serve as powerful encoders for link prediction tasks. The results demonstrate that our DMD-enhanced GNNs achieve state-of-the-art performance, highlighting the effectiveness of integrating DMD into GNN frameworks.

Leveraging the Graph Structure of Neural Network Training Dynamics

How Graph Neural Networks Learn: Lessons from Training Dynamics

DynaGraph: dynamic graph neural networks at scale

Fast and Adaptive Dynamics-on-graphs to Dynamics-of-graphs Translation.

When Graph Neural Networks Meet Dynamic Mode Decomposition

Scaling Up Dynamic Graph Representation Learning Via Spiking Neural Networks

Newton-Cotes Graph Neural Networks: on the Time Evolution of Dynamic Systems

Temporal Graph Networks for Deep Learning on Dynamic Graphs

Learning Dynamics and Structure of Complex Systems Using Graph Neural Networks

ETC: Efficient Training of Temporal Graph Neural Networks over Large-Scale Dynamic Graphs

Exploiting Spiking Dynamics with Spatial-temporal Feature Normalization in Graph Learning

Instant Graph Neural Networks for Dynamic Graphs

Neighborhood Extended Dynamic Graph Neural Network.

Continuous-Time Dynamic Graph Learning Via Neural Interaction Processes.

Temporal Graph Transformer for Dynamic Network

SIMPLE: Efficient Temporal Graph Neural Network Training at Scale with Dynamic Data Placement.

Dynamic Graph Representation Learning With Neural Networks: A Survey

Dynamic Personalized Graph Neural Network with Linear Complexity for Multivariate Time Series Forecasting

DyG2Vec: Efficient Representation Learning for Dynamic Graphs

Efficient and Effective Implicit Dynamic Graph Neural Network

Deep Representations for Time-varying Brain Datasets