Abstract:In many mechanistic medical, biological, physical, and engineered spatiotemporal dynamic models the numerical solution of partial differential equations (PDEs), especially for diffusion, fluid flow and mechanical relaxation, can make simulations impractically slow. Biological models of tissues and organs often require the simultaneous calculation of the spatial variation of concentration of dozens of diffusing chemical species. One clinical example where rapid calculation of a diffusing field is of use is the estimation of oxygen gradients in the retina, based on imaging of the retinal vasculature, to guide surgical interventions in diabetic retinopathy. Furthermore, the ability to predict blood perfusion and oxygenation may one day guide clinical interventions in diverse settings, i.e., from stent placement in treating heart disease to BOLD fMRI interpretation in evaluating cognitive function (Xie et al., 2019; Lee et al., 2020). Since the quasi-steady-state solutions required for fast-diffusing chemical species like oxygen are particularly computationally costly, we consider the use of a neural network to provide an approximate solution to the steady-state diffusion equation. Machine learning surrogates, neural networks trained to provide approximate solutions to such complicated numerical problems, can often provide speed-ups of several orders of magnitude compared to direct calculation. Surrogates of PDEs could enable use of larger and more detailed models than are possible with direct calculation and can make including such simulations in real-time or near-real time workflows practical. Creating a surrogate requires running the direct calculation tens of thousands of times to generate training data and then training the neural network, both of which are computationally expensive. Often the practical applications of such models require thousands to millions of replica simulations, for example for parameter identification and uncertainty quantification, each of which gains speed from surrogate use and rapidly recovers the up-front costs of surrogate generation. We use a Convolutional Neural Network to approximate the stationary solution to the diffusion equation in the case of two equal-diameter, circular, constant-value sources located at random positions in a two-dimensional square domain with absorbing boundary conditions. Such a configuration caricatures the chemical concentration field of a fast-diffusing species like oxygen in a tissue with two parallel blood vessels in a cross section perpendicular to the two blood vessels. To improve convergence during training, we apply a training approach that uses roll-back to reject stochastic changes to the network that increase the loss function. The trained neural network approximation is about 1000 times faster than the direct calculation for individual replicas. Because different applications will have different criteria for acceptable approximation accuracy, we discuss a variety of loss functions and accuracy estimators that can help select the best network for a particular application. We briefly discuss some of the issues we encountered with overfitting, mismapping of the field values and the geometrical conditions that lead to large absolute and relative errors in the approximate solution.

Hybrid Two-level MCMC with Deep Learning Surrogates for Bayesian Inverse Problems

Bayesian Inference and Deep Learning for Inverse Problems

Surrogate-Based Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Structural Error.

An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems

Deep surrogate accelerated delayed-acceptance HMC for Bayesian inference of spatio-temporal heat fluxes in rotating disc systems

Efficient History Matching Using the Markov-Chain Monte Carlo Method by Means of the Transformed Adaptive Stochastic Collocation Method

Adaptive construction of surrogates for the Bayesian solution of inverse problems

Surrogate-Based Bayesian Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Approximation Error

Surrogate modeling for Bayesian inverse problems based on physics-informed neural networks

Applications of ML-Based Surrogates in Bayesian Approaches to Inverse Problems

Adaptive operator learning for infinite-dimensional Bayesian inverse problems

Multilevel Sequential${}^2$ Monte Carlo for Bayesian Inverse Problems

Optimization-Based MCMC Methods for Nonlinear Hierarchical Statistical Inverse Problems

Derivative-informed neural operator acceleration of geometric MCMC for infinite-dimensional Bayesian inverse problems

Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models

Think Twice Before You Act: Improving Inverse Problem Solving With MCMC

Inverse modeling of hydrologic systems with adaptive multi-fidelity Markov chain Monte Carlo simulations

Multilevel Markov Chain Monte Carlo for Bayesian inverse problems for Navier Stokes equation with Lagrangian Observations

Novel Deep neural networks for solving Bayesian statistical inverse

Deep Learning Approaches to Surrogates for Solving the Diffusion Equation for Mechanistic Real-World Simulations

Improving Simulation Efficiency of MCMC for Inverse Modeling of Hydrologic Systems With a Kalman‐Inspired Proposal Distribution