Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

Posterior consistency of Gaussian process prior for nonparametric binary regression

Learning Gaussian Networks

Posterior Concentration for Gaussian Process Priors under Rescaled and Hierarchical Matérn and Confluent Hypergeometric Covariance Functions

LIMIT BEHAVIORS FOR THE INCREMENTS OF A D-Dimensional MULTI-PARAMETER GAUSSIAN PROCESS

Posterior Consistency for Gaussian Process Approximations of Bayesian Posterior Distributions

Advances in Bayesian model selection consistency for high-dimensional generalized linear models

Posterior Covariance Structures in Gaussian Processes

Information consistency of dependent convolved Gaussian processes regression

Posterior Consistency of the Silverman G-Prior in Bayesian Model Choice.

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

Posterior Variance Analysis of Gaussian Processes with Application to Average Learning Curves

Adaptive Bayesian Regression on Data with Low Intrinsic Dimensionality

Contraction rates and projection subspace estimation with Gaussian process priors in high dimension

Contraction rates for conjugate gradient and Lanczos approximate posteriors in Gaussian process regression

Function Optimization with Posterior Gaussian Derivative Process

Introduction To Gaussian Process Regression In Bayesian Inverse Problems, With New ResultsOn Experimental Design For Weighted Error Measures

Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification

Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces

Inference for Gaussian Processes with Matérn Covariogram on Compact Riemannian Manifolds

Gaussian Processes with Errors in Variables: Theory and Computation

Robust Gaussian Process Regression with Huber Likelihood