Abstract:Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an alternative approach, utilizing neural networks combined with ODE solvers to learn continuous latent representations through parameterized vector fields. Neural Stochastic Differential Equations (Neural SDEs) extend Neural ODEs by incorporating a diffusion term, although this addition is not trivial, particularly when addressing irregular intervals and missing values. Consequently, careful design of drift and diffusion functions is crucial for maintaining stability and enhancing performance, while incautious choices can result in adverse properties such as the absence of strong solutions, stochastic destabilization, or unstable Euler discretizations, significantly affecting Neural SDEs' performance. In this study, we propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Then, we rigorously demonstrate their robustness in maintaining excellent performance under distribution shift, while effectively preventing overfitting. To assess the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets for interpolation, forecasting, and classification tasks, and analyze the robustness of our methods with 30 public datasets under different missing rates. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.

Neural ordinary differential equations with irregular and noisy data

Neural ODEs with Irregular and Noisy Data

Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint

Learning Partial Differential Equations from Data Using Neural Networks

Neural Controlled Differential Equations for Irregular Time Series

Learning nonparametric ordinary differential equations from noisy data

Calibrating multi-dimensional complex ODE from noisy data via deep neural networks

Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data

Stiff neural ordinary differential equations

Machine Learning of Partial Differential Equations from Noise Data

Bayesian Neural Ordinary Differential Equations

NeuralPDE: Modelling Dynamical Systems from Data

Invariant Neural Ordinary Differential Equations

ODEN: A Framework to Solve Ordinary Differential Equations using Artificial Neural Networks

Constrained Neural Ordinary Differential Equations with Stability Guarantees

Uncertainty and Structure in Neural Ordinary Differential Equations

Neural Ordinary Differential Equation based Recurrent Neural Network Model

Anamnesic Neural Differential Equations with Orthogonal Polynomial Projections

Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering

Neural Rough Differential Equations for Long Time Series

Collocation based training of neural ordinary differential equations