Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

A Note on the Convergence Rates for Empirical Bayes Estimators of Parameters in Multiple-Parameter Exponential Families

A Supplement to the Complete Convergence

The convergence rates for empirical bayes test of the parameter in scale exponential family

THE CONVERGENCE RATES OF EMPIRICAL BAYES ESTIMATION OF THE PARAMETER FOR SCALE-EXPONENTIAL FAMILY

Convergence rates of empirical Bayesian estimation in a class of linear models

Improvement of convergence rates for the empirical Bayes test problem in scale exponential family

ASYMPTOTICALLY OPTIMAL EMPIRICAL BAYES ESTIMATION FOR THE PARAMETERS OF MULTI-PARAMETER DISCRETE EXPONENTIAL FAMILY

The Convergence Rates of Empirical Bayes Estimation in a Multiple Linear Regression Model

Empirical Bayes Test for Two-parameter Exponential Distribution under Type-II Censored Samples

A Note on the Convergence Rate of the Spectral Distributions of Large Sample Covariance Matrices

CONVERGENCE RATES OF THE EMPIRICAL BAYES TEST PROBLEM FOR CONTINUOUS ONE-PARAMETER EXPONENTIAL FAMILY

RATES OF CONVERGENCE OF M-ESTIMATORS FOR PARTLY LINEAR-MODELS INVOLVING TIME-SERIES

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

The Empirical Bayes Estimation Problem for Continuous One-parameter Exponential Family:In the case of NA Samples

A note on the convergence rate of the kernel density estimator of the mode

Empirical Bayes in Bayesian learning: understanding a common practice

MLE convergence speed to information projection of exponential family: Criterion for model dimension and sample size -- complete proof version--

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

Empirical Bayes Test for Scale Exponential Family

Exponential Convergence in Probability for Empirical Means of Lévy Processes

Empirical Bayes estimates of steady-state availability in exponential distribution model