Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

Convergence Rate of Gibbs Sampler and Its Application

A Supplement to the Complete Convergence

On the Rate of Convergence by Generalized Baskakov Operators

Entropy contraction of the Gibbs sampler under log-concavity

Convergence Rates of Spectral Distributions of Large Sample Covariance Matrices

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Spectral gap bounds for reversible hybrid Gibbs chains

A Note on the Convergence Rate of the Spectral Distributions of Large Sample Covariance Matrices

Adapting The Gibbs Sampler

Markov chain Monte Carlo Methods For Lattice Gaussian Sampling:Convergence Analysis and Enhancement

Analysis of two-component Gibbs samplers using the theory of two projections

Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications

Poisson approximation for two scan statistics with rates of convergence

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

Convergence rate of random scan Coordinate Ascent Variational Inference under log-concavity

Convergence Analysis of Schr{ö}dinger-F{ö}llmer Sampler Without Convexity

Convergence Rate of Multiple-try Metropolis Independent sampler

Adaptive Gibbs samplers

Comparison Theorems for the Mixing Times of Systematic and Random Scan Dynamics

Accelerating Convergence of Replica Exchange Stochastic Gradient MCMC via Variance Reduction

On particle Gibbs sampling