Abstract:We consider the use of randomised forward models and log-likelihoods within the Bayesian approach to inverse problems. Such random approximations to the exact forward model or log-likelihood arise naturally when a computationally expensive model is approximated using a cheaper stochastic surrogate, as in Gaussian process emulation (kriging), or in the field of probabilistic numerical methods. We show that the Hellinger distance between the exact and approximate Bayesian posteriors is bounded by moments of the difference between the true and approximate log-likelihoods. Example applications of these stability results are given for randomised misfit models in large data applications and the probabilistic solution of ordinary differential equations.

What problem does this paper attempt to address?

This paper attempts to solve the problems of using stochastic forward models and log - likelihood functions in Bayesian inverse problems. Specifically, when an accurate but computationally intractable forward model or likelihood function is replaced by a stochastic surrogate model or simulator, the paper studies the stability of the posterior distribution. Such stochastic surrogate models often occur in practice. For example, when an expensive forward model (such as the solution of a PDE) is replaced by a Gaussian process (GP) model, or in big - data applications, high - dimensional residual vectors are randomly sampled or orthogonally projected onto a low - dimensional subspace, or in probabilistic numerical methods, a deterministic dynamical system is solved in a stochastic way that incorporates the apparent uncertainty about the system's behavior. ### Main Contributions 1. **Theoretical Framework**: The paper establishes a theoretical framework for analyzing the impact of stochastic forward models and log - likelihood functions in Bayesian inverse problems. By introducing the Hellinger distance, the author proves that the distance between the true posterior distribution and the approximate posterior distribution can be bounded by the moments of the difference between the true log - likelihood function and the approximate log - likelihood function. 2. **General Results**: The paper is not limited to Gaussian process (GP) models but extends to more general non - Gaussian stochastic approximation models, thus providing broader applicability. 3. **Specific Applications**: The paper provides two specific application examples: - **Stochastic Error Model**: In big - data applications, high - dimensional data become tractable by being projected onto randomly selected low - dimensional subspaces. - **Stochastic Numerical Solution**: In the stochastic numerical solution of a deterministic dynamical system, randomness is used to represent the influence of numerical discretization uncertainty. ### Mathematical Formulas - **Hellinger Distance**: \[ d_H(\mu, \nu)^2=\frac{1}{2} \int_U\left|\sqrt{\frac{d\mu}{d\pi}}-\sqrt{\frac{d\nu}{d\pi}}\right|^2 d\pi = 1-\int_U\sqrt{\frac{d\mu}{d\pi} \frac{d\nu}{d\pi}} d\pi = 1 - E_\nu\left[\sqrt{\frac{d\mu}{d\nu}}\right] \] - **Approximation of Posterior Distribution**: \[ \frac{d\mu^S_N}{d\mu_0}(\omega, u)=\frac{\exp(-\Phi_N(\omega, u))}{Z^S_N(\omega)}, \quad Z^S_N(\omega)=E_{\mu_0}\left[\exp(-\Phi_N(\omega, \cdot))\right] \] \[ \frac{d\mu^M_N}{d\mu_0}(u)=\frac{E_{\nu_N}\left[\exp(-\Phi_N(u))\right]}{E_{\nu_N}\left[Z^S_N\right]} \] ### Conclusion Through rigorous mathematical analysis, the paper proves the stability and convergence of the posterior distribution when using stochastic forward models and log - likelihood functions. These results are of great significance for understanding and applying stochastic surrogate models in Bayesian inverse problems, especially in the fields of big data and numerical computation.

Random forward models and log-likelihoods in Bayesian inverse problems

Bayesian Inference and Deep Learning for Inverse Problems

Choosing observation operators to mitigate model error in Bayesian inverse problems

Generalized Bayes Approach to Inverse Problems with Model Misspecification

Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems

Convergence Analysis of Surrogate-Based Methods for Bayesian Inverse Problems

Generative models and Bayesian inversion using Laplace approximation

A Bayesian Approach to Forward and Inverse Abstract Argumentation Problems

Efficient Variational Bayesian Model Updating by Bayesian Active Learning

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

Stochastic Collocation Algorithms Using L(1)-Minimization for Bayesian Solution of Inverse Problems

On log-concave approximations of high-dimensional posterior measures and stability properties in non-linear inverse problems

Variational Gaussian Processes For Linear Inverse Problems

Posterior exploration for computationally intensive forward models

A variational inference framework for inverse problems

A New Bi‐fidelity Model Reduction Method for Bayesian Inverse Problems

Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces

Introduction To Gaussian Process Regression In Bayesian Inverse Problems, With New ResultsOn Experimental Design For Weighted Error Measures

Generalized sparsity-promoting solvers for Bayesian inverse problems: Versatile sparsifying transforms and unknown noise variances

A posterior contraction for Bayesian inverse problems in Banach spaces

Statistical inverse learning problems with random observations