Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

On the Frequentist Coverage of Bayes Posteriors in Nonlinear Inverse Problems

Statistical guarantees for Bayesian uncertainty quantification in non-linear inverse problems with Gaussian process priors

A posterior contraction for Bayesian inverse problems in Banach spaces

Consistency of variational Bayesian inference for non-linear inverse problems of partial differential equations

Bayesian Posterior Contraction Rates for Linear Severely Ill-posed Inverse Problems

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

Optimization-based frequentist confidence intervals for functionals in constrained inverse problems: Resolving the Burrus conjecture

A Bayesian approach with Gaussian priors to the inverse problem of source identification in elliptic PDEs

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

Convergence Analysis of Surrogate-Based Methods for Bayesian Inverse Problems

Generalized Bayes Approach to Inverse Problems with Model Misspecification

Towards a variational principle for motivated vehicle motion

Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems

Matching Bayesian and frequentist coverage probabilities when using an approximate data covariance matrix

Linear methods for non-linear inverse problems

Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces

Bayesian Approach to Inverse Problems for Functions with A Variable-Index Besov Prior

On some information-theoretic aspects of non-linear statistical inverse problems

Oracle-type Posterior Contraction Rates in Bayesian Inverse Problems

Posterior Contraction for Empirical Bayesian Approach to Inverse Problems under Non-Diagonal Assumption

Consistency of Bayesian inference with Gaussian process priors in an elliptic inverse problem