Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

Some uniform convergence results for kernel estimators

Limit theorems for kernel density estimators under dependent samples

Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension

A Supplement to the Complete Convergence

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

A note on the convergence rate of the kernel density estimator of the mode

Asymptotic Properties of Nonparametric M-Estimation for Mixing Functional Data

Convergence Rates of Spectral Distributions of Large Sample Covariance Matrices

Strong convergence rate of estimators of change point and its application

Kernel regression, minimax rates and effective dimensionality: beyond the regular case

A Note on the Convergence Rate of the Spectral Distributions of Large Sample Covariance Matrices

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Uniform Convergence Rate of the Nonparametric Estimator for Integrated Diffusion Processes

Rates of strong uniform consistency for the $k$-nearest neighbors kernel estimators of density and regression function

Rates of convergence in conditional covariance matrix with nonparametric entries estimation

A High-dimensional Convergence Theorem for U-statistics with Applications to Kernel-based Testing

Malliavin calculus for the optimal estimation of the invariant density of discretely observed diffusions in intermediate regime

A uniform kernel trick for high-dimensional two-sample problems

Asymptotic rates of convergence for compact reversible Markov chains

Convergence Analysis of Kernel Conjugate Gradient for Functional Linear Regression

Statistical Convergence Rates of Optimal Transport Map Estimation between General Distributions