Abstract:Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an alternative approach, utilizing neural networks combined with ODE solvers to learn continuous latent representations through parameterized vector fields. Neural Stochastic Differential Equations (Neural SDEs) extend Neural ODEs by incorporating a diffusion term, although this addition is not trivial, particularly when addressing irregular intervals and missing values. Consequently, careful design of drift and diffusion functions is crucial for maintaining stability and enhancing performance, while incautious choices can result in adverse properties such as the absence of strong solutions, stochastic destabilization, or unstable Euler discretizations, significantly affecting Neural SDEs' performance. In this study, we propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Then, we rigorously demonstrate their robustness in maintaining excellent performance under distribution shift, while effectively preventing overfitting. To assess the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets for interpolation, forecasting, and classification tasks, and analyze the robustness of our methods with 30 public datasets under different missing rates. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.

SDE-HNN: Accurate and Well-calibrated Forecasting using Stochastic Differential Equations

Accurate and Reliable Forecasting using Stochastic Differential Equations

SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates

Neural Stochastic Differential Equations with Neural Processes Family Members for Uncertainty Estimation in Deep Learning

Nonparametric Stochastic Differential Equations for Ultra-Short-Term Probabilistic Forecasting of Wind Power Generation

Milstein-driven neural stochastic differential equation model with uncertainty estimates

Uncertainty Modelling in Deep Networks: Forecasting Short and Noisy Series

A backward SDE method for uncertainty quantification in deep learning

Ensemble Deep Learning-Based Non-Crossing Quantile Regression for Nonparametric Probabilistic Forecasting of Wind Power Generation

The Seven-League Scheme: Deep Learning for Large Time Step Monte Carlo Simulations of Stochastic Differential Equations

Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning.

A Benchmark on Uncertainty Quantification for Deep Learning Prognostics

Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions

Learning effective stochastic differential equations from microscopic simulations: combining stochastic numerics and deep learning

Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data

Data-driven discovery of stochastic differential equations

Accurate Uncertainties for Deep Learning Using Calibrated Regression

Fractional SDE-Net: Generation of Time Series Data with Long-term Memory

STDNet: A Spatio-Temporal Decomposition Neural Network for Multivariate Time Series Forecasting

Efficient and Effective Time-Series Forecasting with Spiking Neural Networks