Abstract:Gaussian process (GP) regression is a fundamental tool in Bayesian statistics. It is also known as kriging and is the Bayesian counterpart to the frequentist kernel ridge regression. Most of the theoretical work on GP regression has focused on a large-$n$ asymptotics, i.e. as the amount of data increases. Fixed-sample analysis is much more difficult outside of simple cases, such as locations on a regular grid. In this work we perform a fixed-sample analysis that was first studied in the context of approximation theory by Driscoll & Fornberg (2002), called the ``flat limit''. In flat-limit asymptotics, the goal is to characterise kernel methods as the length-scale of the kernel function tends to infinity, so that kernels appear flat over the range of the data. Surprisingly, this limit is well-defined, and displays interesting behaviour: Driscoll & Fornberg showed that radial basis interpolation converges in the flat limit to polynomial interpolation, if the kernel is Gaussian. Subsequent work showed that this holds true in the multivariate setting as well, but that kernels other than the Gaussian may have (polyharmonic) splines as the limit interpolant. Leveraging recent results on the spectral behaviour of kernel matrices in the flat limit, we study the flat limit of Gaussian process regression. Results show that Gaussian process regression tends in the flat limit to (multivariate) polynomial regression, or (polyharmonic) spline regression, depending on the kernel. Importantly, this holds for both the predictive mean and the predictive variance, so that the posterior predictive distributions become equivalent. Our results have practical consequences: for instance, they show that optimal GP predictions in the sense of leave-one-out loss may occur at very large length-scales, which would be invisible to current implementations because of numerical difficulties.

Screening Effect in Isotropic Gaussian Processes

When does the screening effect hold?

Local Linear M‐estimation in Non‐parametric Spatial Regression

Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data

Vector-valued Gaussian processes on non-Euclidean product spaces: constructive methods and fast simulations based on partial spectral inversion

Constrained Gaussian Process for Signal Integrity applications using Variational Inference

Spatial Process Approximations: Assessing Their Necessity

Gaussian Process Regression in the Flat Limit

A spatial Gaussian process method for hyperspectral remote sensing imagery classification

Mixed-Stationary Gaussian Process for Flexible Non-Stationary Modeling of Spatial Outcomes

Bias-Free Scalable Gaussian Processes via Randomized Truncations

Further Understanding of a Local Gaussian Process Approximation: Characterising Convergence in the Finite Regime

Radial Neighbors for Provably Accurate Scalable Approximations of Gaussian Processes

Understanding and comparing scalable Gaussian process regression for big data

Inference for Gaussian Processes with Matérn Covariogram on Compact Riemannian Manifolds

Multivariate nearest‐neighbors Gaussian processes with random covariance matrices

Uniform approximation of common Gaussian process kernels using equispaced Fourier grids

Asymptotic properties of the maximum likelihood and cross validation estimators for transformed Gaussian processes

When Gaussian Process Meets Big Data: A Review of Scalable GPs

Balancing the edge effect and dimension of spectral spatial statistics under irregular sampling with applications to isotropy testing