Abstract:The Gaussian process (GP) is a widely used probabilistic machine learning method for stochastic function approximation, stochastic modeling, and analyzing real-world measurements of nonlinear processes. Unlike many other machine learning methods, GPs include an implicit characterization of uncertainty, making them extremely useful across many areas of science, technology, and engineering. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility and exact methods for inference that prevent application to data sets with more than about ten thousand points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space-time prediction based on more than one million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.

Sketching the Heat Kernel: Using Gaussian Processes to Embed Data

Easy representation of multivariate functions with low-dimensional terms via Gaussian process regression kernel design: applications to machine learning of potential energy surfaces and kinetic energy densities from sparse data

High-Dimensional Gaussian Process Regression with Soft Kernel Interpolation

Scalable Bayesian inference for heat kernel Gaussian processes on manifolds

Vector-valued Gaussian processes on non-Euclidean product spaces: constructive methods and fast simulations based on partial spectral inversion

Gradient Sketches for Training Data Attribution and Studying the Loss Landscape

Learning Manifold Implicitly via Explicit Heat-Kernel Learning

Thin and Deep Gaussian Processes

The GeometricKernels Package: Heat and Matérn Kernels for Geometric Learning on Manifolds, Meshes, and Graphs

Sparse Gaussian Processes with Spherical Harmonic Features Revisited

Compactly-supported nonstationary kernels for computing exact Gaussian processes on big data

Intrinsic Gaussian Process on Unknown Manifolds with Probabilistic Metrics

A Unifying Perspective on Non-Stationary Kernels for Deeper Gaussian Processes

Uniform approximation of common Gaussian process kernels using equispaced Fourier grids

Statistical properties of sketching algorithms

Heat content for Gaussian processes: small-time asymptotic analysis

The Parametrix Construction of the Heat Kernel on a Graph

High-dimensional Gaussian linear processes: Marchenko-Pastur beyond simultaneous diagonalizability

Wiener Chaos in Kernel Regression: Towards Untangling Aleatoric and Epistemic Uncertainty

Sparse Orthogonal Variational Inference for Gaussian Processes