Abstract:Predicting the trajectories of systems with unknown dynamics (\textit{i.e.} the governing rules) is crucial in various research fields, including physics and biology. This challenge has gathered significant attention from diverse communities. Most existing works focus on learning fixed system dynamics within one single system. However, real-world applications often involve multiple systems with different types of dynamics or evolving systems with non-stationary dynamics (dynamics shifts). When data from those systems are continuously collected and sequentially fed to machine learning models for training, these models tend to be biased toward the most recently learned dynamics, leading to catastrophic forgetting of previously observed/learned system dynamics. To this end, we aim to learn system dynamics via continual learning. Specifically, we present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics and encode the system-specific dynamics into binary masks over the model parameters. During the inference stage, the model can select the most confident mask based on the observational data to identify the system and predict future trajectories accordingly. Empirically, we systematically investigate the task configurations and compare the proposed MS-GODE with state-of-the-art techniques. More importantly, we construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics and significantly enriching the research field of machine learning for dynamic systems.

Learn from one and predict all: single trajectory learning for time delay systems

Adaptive Exponential Synchronization of Multislave Time-Delayed Recurrent Neural Networks with Lévy Noise and Regime Switching

Trainable Delays in Time Delay Neural Networks for Learning Delayed Dynamics

Learning the Delay Using Neural Delay Differential Equations

Trajectory Forecasting with Neural Networks: an Empirical Evaluation and A New Hybrid Model

Exponential State Estimator Design for Discrete-Time Neural Networks with Discrete and Distributed Time-Varying Delays.

Exponential Synchronization of Discrete-Time Chaotic Delayed Neural Networks

Lyapunov Functional for Neural Networks with Delayed Interactions and Statistical Mechanics of Temporal Associations

Joint Problems in Learning Multiple Dynamical Systems

Dynamical stability and chaos in artificial neural network trajectories along training

Neural DDEs with Learnable Delays for Partially Observed Dynamical Systems

Guaranteed Trajectory Tracking under Learned Dynamics with Contraction Metrics and Disturbance Estimation

Universal Approach to Study Delayed Dynamical Systems

Optimal exponential synchronization of general chaotic delayed neural networks: an LMI approach

Recovering discrete delayed fractional equations from trajectories

Real-Time Variational Method for Learning Neural Trajectory and its Dynamics

Exponential Stability of Neural Networks with Variable Delays Via LMI Approach

Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systems

Learning System Dynamics without Forgetting

Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning

Modeling Trajectories with Neural Ordinary Differential Equations