Abstract:Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an alternative approach, utilizing neural networks combined with ODE solvers to learn continuous latent representations through parameterized vector fields. Neural Stochastic Differential Equations (Neural SDEs) extend Neural ODEs by incorporating a diffusion term, although this addition is not trivial, particularly when addressing irregular intervals and missing values. Consequently, careful design of drift and diffusion functions is crucial for maintaining stability and enhancing performance, while incautious choices can result in adverse properties such as the absence of strong solutions, stochastic destabilization, or unstable Euler discretizations, significantly affecting Neural SDEs' performance. In this study, we propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Then, we rigorously demonstrate their robustness in maintaining excellent performance under distribution shift, while effectively preventing overfitting. To assess the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets for interpolation, forecasting, and classification tasks, and analyze the robustness of our methods with 30 public datasets under different missing rates. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.

Neural ODEs with Irregular and Noisy Data

Neural ordinary differential equations with irregular and noisy data

Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint

Learning Partial Differential Equations from Data Using Neural Networks

Calibrating multi-dimensional complex ODE from noisy data via deep neural networks

Neural Controlled Differential Equations for Irregular Time Series

Learning nonparametric ordinary differential equations from noisy data

Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data

NeuralPDE: Modelling Dynamical Systems from Data

Bayesian Neural Ordinary Differential Equations

Stiff neural ordinary differential equations

Invariant Neural Ordinary Differential Equations

ODEN: A Framework to Solve Ordinary Differential Equations using Artificial Neural Networks

Noisy Learning for Neural ODEs Acts as a Robustness Locus Widening

Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering

Uncertainty and Structure in Neural Ordinary Differential Equations

Beyond Predictions in Neural ODEs: Identification and Interventions.

Neural Ordinary Differential Equation based Recurrent Neural Network Model

Learning Governing Equations of Unobserved States in Dynamical Systems

Constrained Neural Ordinary Differential Equations with Stability Guarantees

Neural Flows: Efficient Alternative to Neural ODEs