Abstract:Inverse problems, i.e., estimating parameters of physical models from experimental data, are ubiquitous in science and engineering. The Bayesian formulation is the gold standard because it alleviates ill-posedness issues and quantifies epistemic uncertainty. Since analytical posteriors are not typically available, one resorts to Markov chain Monte Carlo sampling or approximate variational inference. However, inference needs to be rerun from scratch for each new set of data. This drawback limits the applicability of the Bayesian formulation to real-time settings, e.g., health monitoring of engineered systems, and medical diagnosis. The objective of this paper is to develop a methodology that enables real-time inference by learning the Bayesian inverse map, i.e., the map from data to posteriors. Our approach is as follows. We parameterize the posterior distribution as a function of data. This work outlines two distinct approaches to do this. The first method involves parameterizing the posterior using an amortized full-rank Gaussian guide, implemented through neural networks. The second method utilizes a Conditional Normalizing Flow guide, employing conditional invertible neural networks for cases where the target posterior is arbitrarily complex. In both approaches, we learn the network parameters by amortized variational inference which involves maximizing the expectation of evidence lower bound over all possible datasets compatible with the model. We demonstrate our approach by solving a set of benchmark problems from science and engineering. Our results show that the posterior estimates of our approach are in agreement with the corresponding ground truth obtained by Markov chain Monte Carlo. Once trained, our approach provides the posterior distribution for a given observation just at the cost of a forward pass of the neural network.

Towards a variational principle for motivated vehicle motion

Bayesian Inference and Deep Learning for Inverse Problems

Non-intrusive optimal experimental design for large-scale nonlinear Bayesian inverse problems using a Bayesian approximation error approach

A posterior contraction for Bayesian inverse problems in Banach spaces

Optimal design of large-scale nonlinear Bayesian inverse problems under model uncertainty

Generalized Bayes Approach to Inverse Problems with Model Misspecification

A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion

Efficient hyperparameter estimation in Bayesian inverse problems using sample average approximation

Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides

Adaptive construction of surrogates for the Bayesian solution of inverse problems

Inexact Generalized Golub-Kahan Methods for Large-Scale Bayesian Inverse Problems

Variational Gaussian Processes For Linear Inverse Problems

Analysis of a Computational Framework for Bayesian Inverse Problems: Ensemble Kalman Updates and MAP Estimators Under Mesh Refinement

Bayesian Approach to Inverse Problems for Functions with A Variable-Index Besov Prior

Recent advances in Bayesian optimization with applications to parameter reconstruction in optical nano-metrology

Posterior Contraction for Empirical Bayesian Approach to Inverse Problems under Non-Diagonal Assumption

Linear methods for non-linear inverse problems

Adaptive operator learning for infinite-dimensional Bayesian inverse problems

On the Frequentist Coverage of Bayes Posteriors in Nonlinear Inverse Problems