Abstract:Gaussian process (GP) regression is a fundamental tool in Bayesian statistics. It is also known as kriging and is the Bayesian counterpart to the frequentist kernel ridge regression. Most of the theoretical work on GP regression has focused on a large-$n$ asymptotics, i.e. as the amount of data increases. Fixed-sample analysis is much more difficult outside of simple cases, such as locations on a regular grid. In this work we perform a fixed-sample analysis that was first studied in the context of approximation theory by Driscoll & Fornberg (2002), called the ``flat limit''. In flat-limit asymptotics, the goal is to characterise kernel methods as the length-scale of the kernel function tends to infinity, so that kernels appear flat over the range of the data. Surprisingly, this limit is well-defined, and displays interesting behaviour: Driscoll & Fornberg showed that radial basis interpolation converges in the flat limit to polynomial interpolation, if the kernel is Gaussian. Subsequent work showed that this holds true in the multivariate setting as well, but that kernels other than the Gaussian may have (polyharmonic) splines as the limit interpolant. Leveraging recent results on the spectral behaviour of kernel matrices in the flat limit, we study the flat limit of Gaussian process regression. Results show that Gaussian process regression tends in the flat limit to (multivariate) polynomial regression, or (polyharmonic) spline regression, depending on the kernel. Importantly, this holds for both the predictive mean and the predictive variance, so that the posterior predictive distributions become equivalent. Our results have practical consequences: for instance, they show that optimal GP predictions in the sense of leave-one-out loss may occur at very large length-scales, which would be invisible to current implementations because of numerical difficulties.

Gaussian and sparse processes are limits of generalized Poisson processes

On Moduli of Continuity for Gaussian and L2-Norm Squared Processes Generated by Ornstein-Uhlenbeck Processes

On Bounded Stochastic Processes

On infinite series of independent Ornstein-Uhlenbeck processes

Some Limit Results for the Multi-parameter Gaussian Processes with Stationary Increments

LIMIT BEHAVIORS FOR THE INCREMENTS OF A D-Dimensional MULTI-PARAMETER GAUSSIAN PROCESS

On the fractal nature of increments of lp-valued Gaussian processes

On large increments and liminfs of a two-parameter Gaussian process

Generalised shot-noise representations of stochastic systems driven by non-Gaussian Lévy processes

Generalised shot noise representations of stochastic systems driven by non-Gaussian Lévy processes

Poisson approximation for stochastic processes summed over amenable groups

Scattering Statistics of Generalized Spatial Poisson Point Processes

Poisson limit theorem for the number of excursions above high and medium levels by Gaussian stationary sequences

Gaussian Approximation and Moderate Deviations of Poisson Shot Noises with Application to Compound Generalized Hawkes Processes

A Generalized Stochastic Process for Count Data

Poisson Processes in Free Probability

Images of Gaussian and other stochastic processes under closed, densely-defined, unbounded linear operators

Local weak convergence for sparse networks of interacting processes

Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility

Gaussian Process Regression in the Flat Limit

Variational Gaussian Process Diffusion Processes