Abstract:We present a novel variational framework for performing inference in (neural) stochastic differential equations (SDEs) driven by Markov-approximate fractional Brownian motion (fBM). SDEs offer a versatile tool for modeling real-world continuous-time dynamic systems with inherent noise and randomness. Combining SDEs with the powerful inference capabilities of variational methods, enables the learning of representative function distributions through stochastic gradient descent. However, conventional SDEs typically assume the underlying noise to follow a Brownian motion (BM), which hinders their ability to capture long-term dependencies. In contrast, fractional Brownian motion (fBM) extends BM to encompass non-Markovian dynamics, but existing methods for inferring fBM parameters are either computationally demanding or statistically inefficient. In this paper, building upon the Markov approximation of fBM, we derive the evidence lower bound essential for efficient variational inference of posterior path measures, drawing from the well-established field of stochastic analysis. Additionally, we provide a closed-form expression to determine optimal approximation coefficients. Furthermore, we propose the use of neural networks to learn the drift, diffusion and control terms within our variational posterior, leading to the variational training of neural-SDEs. In this framework, we also optimize the Hurst index, governing the nature of our fractional noise. Beyond validation on synthetic data, we contribute a novel architecture for variational latent video prediction,-an approach that, to the best of our knowledge, enables the first variational neural-SDE application to video perception.

Statistical Mechanics of Learning: A Variational Approach for Real Data

Data-Driven Automated Discovery of Variational Laws Hidden in Physical Systems

Solving Statistical Mechanics using Variational Autoregressive Networks

Learning Generalized Statistical Mechanics with Matrix Product States

Variational Learning of Euler-Lagrange Dynamics from Data

Statistical Physics through the Lens of Real-Space Mutual Information

Variational Inference for Stochastic Differential Equations

Variational mcmc

Understanding Machine Learning Paradigms through the Lens of Statistical Thermodynamics: A tutorial

Variational Bayesian inference with stochastic search

Entropy, concentration, and learning: a statistical mechanics primer

Variational Evolutionary Network for Statistical Physics Systems

An approach to non-equilibrium statistical physics using variational Bayesian inference

Variational learning for switching state-space models

Variational methods for the Dirichlet process

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Variational Monte Carlo - Bridging Concepts of Machine Learning and High Dimensional Partial Differential Equations

Variational Inference for SDEs Driven by Fractional Noise

Sparse inference and active learning of stochastic differential equations from data

Statistical Mechanics Calculations Using Variational Autoregressive Networks and Quantum Annealing

Variational Inference: A Review for Statisticians