Abstract:Learning multi-agent system dynamics has been extensively studied for various real-world applications, such as molecular dynamics in biology. Most of the existing models are built to learn single system dynamics from observed historical data and predict the future trajectory. In practice, however, we might observe multiple systems that are generated across different environments, which differ in latent exogenous factors such as temperature and gravity. One simple solution is to learn multiple environment-specific models, but it fails to exploit the potential commonalities among the dynamics across environments and offers poor prediction results where per-environment data is sparse or limited. Here, we present GG-ODE (Generalized Graph Ordinary Differential Equations), a machine learning framework for learning continuous multi-agent system dynamics across environments. Our model learns system dynamics using neural ordinary differential equations (ODE) parameterized by Graph Neural Networks (GNNs) to capture the continuous interaction among agents. We achieve the model generalization by assuming the dynamics across different environments are governed by common physics laws that can be captured via learning a shared ODE function. The distinct latent exogenous factors learned for each environment are incorporated into the ODE function to account for their differences. To improve model performance, we additionally design two regularization losses to (1) enforce the orthogonality between the learned initial states and exogenous factors via mutual information minimization; and (2) reduce the temporal variance of learned exogenous factors within the same system via contrastive learning. Experiments over various physical simulations show that our model can accurately predict system dynamics, especially in the long range, and can generalize well to new systems with few observations.

Stochastic Neural Simulator for Generalizing Dynamical Systems Across Environments

Noise-aware neural network for stochastic dynamics simulation

Learning continuous network emerging dynamics from scarce observations via data-adaptive stochastic processes

Generalizing Graph ODE for Learning Complex System Dynamics across Environments

Sparsity in Continuous-Depth Neural Networks

Equivariant Neural Simulators for Stochastic Spatiotemporal Dynamics

LEADS: Learning Dynamical Systems that Generalize Across Environments

CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation

How to Learn and Generalize From Three Minutes of Data: Physics-Constrained and Uncertainty-Aware Neural Stochastic Differential Equations

Modeling Spatio-temporal Dynamical Systems with Neural Discrete Learning and Levels-of-Experts

Towards Complex Dynamic Physics System Simulation with Graph Neural ODEs

ControlSynth Neural ODEs: Modeling Dynamical Systems with Guaranteed Convergence

Deep Generative Modeling for Identification of Noisy, Non-Stationary Dynamical Systems

Neural Downscaling for Complex Systems: from Large-scale to Small-scale by Neural Operator

Neural Context Flows for Meta-Learning of Dynamical Systems

Stochastic physics-informed neural ordinary differential equations

Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs

NeuralPDE: Modelling Dynamical Systems from Data

Neural Stochastic Differential Equations with Neural Processes Family Members for Uncertainty Estimation in Deep Learning

Learning to Simulate Complex Physical Systems: A Case Study