Abstract:Inverse problems, i.e., estimating parameters of physical models from experimental data, are ubiquitous in science and engineering. The Bayesian formulation is the gold standard because it alleviates ill-posedness issues and quantifies epistemic uncertainty. Since analytical posteriors are not typically available, one resorts to Markov chain Monte Carlo sampling or approximate variational inference. However, inference needs to be rerun from scratch for each new set of data. This drawback limits the applicability of the Bayesian formulation to real-time settings, e.g., health monitoring of engineered systems, and medical diagnosis. The objective of this paper is to develop a methodology that enables real-time inference by learning the Bayesian inverse map, i.e., the map from data to posteriors. Our approach is as follows. We parameterize the posterior distribution as a function of data. This work outlines two distinct approaches to do this. The first method involves parameterizing the posterior using an amortized full-rank Gaussian guide, implemented through neural networks. The second method utilizes a Conditional Normalizing Flow guide, employing conditional invertible neural networks for cases where the target posterior is arbitrarily complex. In both approaches, we learn the network parameters by amortized variational inference which involves maximizing the expectation of evidence lower bound over all possible datasets compatible with the model. We demonstrate our approach by solving a set of benchmark problems from science and engineering. Our results show that the posterior estimates of our approach are in agreement with the corresponding ground truth obtained by Markov chain Monte Carlo. Once trained, our approach provides the posterior distribution for a given observation just at the cost of a forward pass of the neural network.

Convergence of Gaussian Process Regression with Estimated Hyper-Parameters and Applications in Bayesian Inverse Problems

Convergence rates of non-stationary and deep Gaussian process regression

Introduction To Gaussian Process Regression In Bayesian Inverse Problems, With New ResultsOn Experimental Design For Weighted Error Measures

Posterior Concentration for Gaussian Process Priors under Rescaled and Hierarchical Matérn and Confluent Hypergeometric Covariance Functions

Consistency of Bayesian inference with Gaussian process priors for a parabolic inverse problem

Posterior Consistency for Gaussian Process Approximations of Bayesian Posterior Distributions

Iterative Construction of Gaussian Process Surrogate Models for Bayesian Inference

Adaptive Gaussian Process Regression for Bayesian inverse problems

An adaptive Gaussian process method for multi-modal Bayesian inverse problems

Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Models

On the Inference of Applying Gaussian Process Modeling to a Deterministic Function

Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides

Contraction rates for conjugate gradient and Lanczos approximate posteriors in Gaussian process regression

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent

Consistency of Bayesian inference with Gaussian process priors in an elliptic inverse problem

ANOVA-GP Modeling for High-Dimensional Bayesian Inverse Problems

Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification

Variational Gaussian Processes For Linear Inverse Problems

Efficient hyperparameter estimation in Bayesian inverse problems using sample average approximation

Posterior sampling with Adaptive Gaussian Processes in Bayesian parameter identification