Abstract:We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Bayesian inference on millions of uncertain network parameters. This approach achieves state of the art performance in terms of predictive accuracy and uncertainty quantification in comparison to other approaches in Bayesian neural networks as well as techniques that include Gaussian processes and ensemble methods even when the training data size is relatively small. To evaluate the performance of this approach, we consider standard uncertainty quantification benchmark problems including flow in heterogeneous media defined in terms of limited data-driven permeability realizations. The performance of the surrogate model developed is very good even though there is no underlying structure shared between the input (permeability) and output (flow/pressure) fields as is often the case in the image-to-image regression models used in computer vision problems. Studies are performed with an underlying stochastic input dimensionality up to $4,225$ where most other uncertainty quantification methods fail. Uncertainty propagation tasks are considered and the predictive output Bayesian statistics are compared to those obtained with Monte Carlo estimates.

Milstein-driven neural stochastic differential equation model with uncertainty estimates

SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates

A backward SDE method for uncertainty quantification in deep learning

Accurate and Reliable Forecasting using Stochastic Differential Equations

Neural Stochastic Differential Equations with Neural Processes Family Members for Uncertainty Estimation in Deep Learning

Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise

Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning.

SDE-HNN: Accurate and Well-calibrated Forecasting using Stochastic Differential Equations

A deep learning-based Monte Carlo simulation scheme for stochastic differential equations driven by fractional Brownian motion

The Seven-League Scheme: Deep Learning for Large Time Step Monte Carlo Simulations of Stochastic Differential Equations

Learning effective stochastic differential equations from microscopic simulations: combining stochastic numerics and deep learning

Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations

Error Bounds for Deep Learning-based Uncertainty Propagation in SDEs

Functional SDE approximation inspired by a deep operator network architecture

A General Framework for Uncertainty Quantification via Neural SDE-RNN

Neural stochastic differential equations network as uncertainty quantification method for EEG source localization

Uncertainty quantification for deep learning-based schemes for solving high-dimensional backward stochastic differential equations

Asymptotic Error for the Milstein Scheme for SDEs with Stochastic Evaluation Times

Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions

Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification