Abstract:Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an alternative approach, utilizing neural networks combined with ODE solvers to learn continuous latent representations through parameterized vector fields. Neural Stochastic Differential Equations (Neural SDEs) extend Neural ODEs by incorporating a diffusion term, although this addition is not trivial, particularly when addressing irregular intervals and missing values. Consequently, careful design of drift and diffusion functions is crucial for maintaining stability and enhancing performance, while incautious choices can result in adverse properties such as the absence of strong solutions, stochastic destabilization, or unstable Euler discretizations, significantly affecting Neural SDEs' performance. In this study, we propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Then, we rigorously demonstrate their robustness in maintaining excellent performance under distribution shift, while effectively preventing overfitting. To assess the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets for interpolation, forecasting, and classification tasks, and analyze the robustness of our methods with 30 public datasets under different missing rates. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.

Neural stochastic Volterra equations: learning path-dependent dynamics

Learning stochastic dynamical systems with neural networks mimicking the Euler-Maruyama scheme

Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning.

Neural Laplace for learning Stochastic Differential Equations

Learning Continuous-Time Dynamics by Stochastic Differential Networks

Learning effective stochastic differential equations from microscopic simulations: combining stochastic numerics and deep learning

Improving the Noise Estimation of Latent Neural Stochastic Differential Equations

Stochastic physics-informed neural ordinary differential equations

How to Learn and Generalize From Three Minutes of Data: Physics-Constrained and Uncertainty-Aware Neural Stochastic Differential Equations

Generative Modeling of Neural Dynamics via Latent Stochastic Differential Equations

Neural Jump Stochastic Differential Equations

Equivariant Neural Simulators for Stochastic Spatiotemporal Dynamics

Neural variance reduction for stochastic differential equations

Variational Neural Stochastic Differential Equations with Change Points

Random neural networks for rough volatility

Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data

A Neural Stochastic Volatility Model

Stiff neural ordinary differential equations

Well-posedness and stability of a stochastic neural field in the form of a partial differential equation

Neural network stochastic differential equation models with applications to financial data forecasting

Autonomous learning of nonlocal stochastic neuron dynamics