Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

Kernel Estimation of Density Level Sets

Limit theorems for kernel density estimators under dependent samples

ASYMPTOTIC NORMALITY OF KERNEL ESTIMATES OF A DENSITY FUNCTION UNDER ASSOCIATION DEPENDENCE

Level set and density estimation on manifolds

Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension

Consistency of Kernel Density Estimators for Causal Processes

EMPIRICAL LIKELIHOOD‐BASED KERNEL DENSITY ESTIMATION

Beta Kernel Estimators for Density Functions

Rates of convergence in conditional covariance matrix with nonparametric entries estimation

Probability Density Function Estimation Using Gamma Kernels

On the Estimation of Derivatives Using Plug-in Kernel Ridge Regression Estimators

Convolutions of Totally Positive Distributions with applications to Kernel Density Estimation

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

Rates of strong uniform consistency for the $k$-nearest neighbors kernel estimators of density and regression function

Kernel density estimation with polyspherical data and its applications

Robust Sparse Kernel Density Estimation by Inducing Randomness

Nonparametric kernel estimation of conditional copula density

Uniform Inference for Kernel Density Estimators with Dyadic Data

Fast multivariate empirical cumulative distribution function with connection to kernel density estimation

Kernel Density Estimation through Density Constrained Near Neighbor Search

Kernel estimation of multivariate cumulative distribution function