Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

Oracle-type Posterior Contraction Rates in Bayesian Inverse Problems

A posterior contraction for Bayesian inverse problems in Banach spaces

Consistency of the Bayes method for the inverse scattering problem

Strong posterior contraction rates via Wasserstein dynamics

Posterior contraction rates in a sparse non-linear mixed-effects model

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

Generalized Bayes Approach to Inverse Problems with Model Misspecification

On the Frequentist Coverage of Bayes Posteriors in Nonlinear Inverse Problems

Consistency of Bayesian inference with Gaussian process priors for a parabolic inverse problem

Bayesian inverse problems with $l_1$ priors: a Randomize-then-Optimize approach

Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems

Posterior Contraction Rates for Deconvolution of Dirichlet-Laplace Mixtures

Application of Bayesian Inference to Estimate the Parameters in 2D Convection-diffusion Equation with Source

On strong posterior contraction rates for Besov-Laplace priors in the white noise model

Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces

Posterior Concentration for Gaussian Process Priors under Rescaled and Hierarchical Matérn and Confluent Hypergeometric Covariance Functions

Inexact Generalized Golub-Kahan Methods for Large-Scale Bayesian Inverse Problems

A Bayesian approach with Gaussian priors to the inverse problem of source identification in elliptic PDEs

Choosing observation operators to mitigate model error in Bayesian inverse problems

Variational Gaussian Processes For Linear Inverse Problems

Inverse Problems in a Bayesian Setting