Abstract:The major challenges in the Bayesian inverse problems arise from the need for repeated evaluations of the forward model, as required by Markov chain Monte Carlo (MCMC) methods for posterior sampling. Many attempts at accelerating Bayesian inference have relied on surrogates for the forward model, typically constructed through repeated forward simulations that are performed in an offline phase. Although such approaches can be quite effective at reducing computation cost, there has been little analysis of the approximation on posterior inference. In this work, we prove error bounds on the Kullback-Leibler (KL) distance between the true posterior distribution and the approximation based on surrogate models. Our rigorous error analysis show that if the forward model approximation converges at certain rate in the prior-weighted L-2 norm, then the posterior distribution generated by the approximation converges to the true posterior at least two times faster in the KL sense. The error bound on the Hellinger distance is also provided. To provide concrete examples focusing on the use of the surrogate model based methods, we present an efficient technique for constructing stochastic surrogate models to accelerate the Bayesian inference approach. The Christoffel least squares algorithms, based on generalized polynomial chaos, are used to construct a polynomial approximation of the forward solution over the support of the prior distribution. The numerical strategy and the predicted convergence rates are then demonstrated on the nonlinear inverse problems, involving the inference of parameters appearing in partial differential equations.

Constrained inference through posterior projections

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Bayesian Inference Using the Proximal Mapping: Uncertainty Quantification Under Varying Dimensionality

High-dimensional Grouped-regression using Bayesian Sparse Projection-posterior

Maximum-a-posteriori estimation with Bayesian confidence regions

Learning Bayesian Networks by Constrained Bayesian Estimation

Proximal MCMC for Bayesian Inference of Constrained and Regularized Estimation

Post-Processed Posteriors for Banded Covariances

Understanding posterior projection effects with normalizing flows

Mixed-Projection Conic Optimization: A New Paradigm for Modeling Rank Constraints

Inference for a constrained parameter in presence of an uncertain constraint

A posterior contraction for Bayesian inverse problems in Banach spaces

On Degrees of Freedom of Projection Estimators with Applications to Multivariate Shape Restricted Regression

The Bridged Posterior: Optimization, Profile Likelihood and a New Approach to Generalized Bayes

Adding Constraints to Bayesian Inverse Problems.

Properly constrained reference priors decay rates for efficient and robust posterior inference

Convergence Analysis of Surrogate-Based Methods for Bayesian Inverse Problems

Universal Inference Meets Random Projections: A Scalable Test for Log-concavity

Optimal prior for Bayesian inference in a constrained parameter space

A projection approach for multiple monotone regression

Bayesian nonparametric multivariate convex regression