Abstract:We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to the order of derivatives and enjoys a generalized plug-in property that extends real-valued functionals to function-valued functionals. This leads to a practically simple method for estimating the regression function and its derivatives, whose finite sample performance is assessed using simulations. Our proof shows that, under certain conditions, to any convergence rate of Bayes estimators there corresponds the same convergence rate of the posterior distributions (i.e., posterior contraction rate), and vice versa. This equivalence holds for a general class of Gaussian processes and covers the regression function and its derivative functionals, under both the $L_2$ and $L_{\infty}$ norms. In addition to connecting these two fundamental large sample properties in Bayesian and non-Bayesian regimes, such equivalence enables a new routine to establish posterior contraction rates by calculating convergence rates of nonparametric point estimators. At the core of our argument is an operator-theoretic framework for kernel ridge regression and equivalent kernel techniques. We derive a range of sharp non-asymptotic bounds that are pivotal in establishing convergence rates of nonparametric point estimators and the equivalence theory, which may be of independent interest.

Posterior Contraction Rates for Deconvolution of Dirichlet-Laplace Mixtures

Bayesian estimation and prediction for certain mixtures

Optimal Reduction of Multivariate Dirac Mixture Densities

Continuous Gaussian mixture solution for linear Bayesian inversion with application to Laplace priors

Adaptive Bayesian multivariate density estimation with Dirichlet mixtures

Posterior contraction rates in a sparse non-linear mixed-effects model

Oracle-type Posterior Contraction Rates in Bayesian Inverse Problems

Mixture representations and Bayesian nonparametric inference for likelihood ratio ordered distributions

Bayesian inversion with Student's t priors based on Gaussian scale mixtures

A posterior contraction for Bayesian inverse problems in Banach spaces

Nearest Neighbor Dirichlet Mixtures

Adaptive nonparametric Bayesian inference using location-scale mixture priors

Adaptive Bayesian density estimation using Pitman-Yor or normalized inverse-Gaussian process kernel mixtures

Mixed Laplace approximation for marginal posterior and Bayesian inference in error-in-operator model

On strong posterior contraction rates for Besov-Laplace priors in the white noise model

Strong posterior contraction rates via Wasserstein dynamics

Convergence of de Finetti's mixing measure in latent structure models for observed exchangeable sequences

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

Bayesian nonparametric location-scale-shape mixtures

Mixture Models With a Prior on the Number of Components