Abstract:Recently, Neural Ordinary Differential Equations has emerged as a powerful framework for modeling physical simulations without explicitly defining the ODEs governing the system, but instead learning them via machine learning. However, the question: "Can Bayesian learning frameworks be integrated with Neural ODE's to robustly quantify the uncertainty in the weights of a Neural ODE?" remains unanswered. In an effort to address this question, we primarily evaluate the following categories of inference methods: (a) The No-U-Turn MCMC sampler (NUTS), (b) Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) and (c) Stochastic Langevin Gradient Descent (SGLD). We demonstrate the successful integration of Neural ODEs with the above Bayesian inference frameworks on classical physical systems, as well as on standard machine learning datasets like MNIST, using GPU acceleration. On the MNIST dataset, we achieve a posterior sample accuracy of 98.5% on the test ensemble of 10,000 images. Subsequently, for the first time, we demonstrate the successful integration of variational inference with normalizing flows and Neural ODEs, leading to a powerful Bayesian Neural ODE object. Finally, considering a predator-prey model and an epidemiological system, we demonstrate the probabilistic identification of model specification in partially-described dynamical systems using universal ordinary differential equations. Together, this gives a scientific machine learning tool for probabilistic estimation of epistemic uncertainties.

Bayesian topology inference on partially known networks from input-output pairs

Bayesian topology identification of linear dynamic networks

Learning Gaussian Networks

Bayesian inference of transition matrices from incomplete graph data with a topological prior

A Full Bayesian Approach to Sparse Network Inference Using Heterogeneous Datasets

Joint Inference Of Networks From Stationary Graph Signals

System identification using Bayesian neural networks with nonparametric noise models

Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks?

Using a Bayesian approach to reconstruct graph statistics after edge sampling

Bayesian learning of multiple directed networks from observational data

Forecasting-aided Joint Topology/State Estimation for Distribution Systems via Bayesian Non-parametric Modeling

Top-Down Bayesian Posterior Sampling for Sum-Product Networks

A Bayesian approach to parameter identification with an application to Turing systems

Inferring the Graph of Networked Dynamical Systems under Partial Observability and Spatially Colored Noise

Estimation of partially known Gaussian graphical models with score-based structural priors

Bayesian Inference of Networks Across Multiple Sample Groups and Data Types

Joint Network Topology Inference in the Presence of Hidden Nodes

Inferring networks from time series: a neural approach

Bayesian networks in neuroscience: a survey

Bayesian Neural Ordinary Differential Equations

Gibbs sampling the posterior of neural networks