Abstract:The Gaussian process (GP) is a widely used probabilistic machine learning method for stochastic function approximation, stochastic modeling, and analyzing real-world measurements of nonlinear processes. Unlike many other machine learning methods, GPs include an implicit characterization of uncertainty, making them extremely useful across many areas of science, technology, and engineering. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility and exact methods for inference that prevent application to data sets with more than about ten thousand points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space-time prediction based on more than one million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.

Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior

Sparsity-Aware Distributed Learning for Gaussian Processes with Linear Multiple Kernel

Composite Gaussian Processes: Scalable Computation and Performance Analysis

Adaptive Sparse Gaussian Process

Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression

Compressing spectral kernels in Gaussian Process: Enhanced generalization and interpretability

Structured Variational Inference for Coupled Gaussian Processes

Representing Additive Gaussian Processes by Sparse Matrices

Nonparametric Bayesian Mixed-effect Model: a Sparse Gaussian Process Approach

Sparse Multimodal Gaussian Processes

Multiview Variational Sparse Gaussian Processes.

Compressible Spectral Mixture Kernels with Sparse Dependency Structures for Gaussian Processes

Sparse Gaussian Processes with Manifold-Preserving Graph Reduction

Hierarchical Shrinkage Gaussian Processes: Applications to Computer Code Emulation and Dynamical System Recovery

ProSpar-GP: scalable Gaussian process modeling with massive non-stationary datasets

Spike and Slab Gaussian Process Latent Variable Models

Sparse shape composition: A new framework for shape prior modeling

EigenGP: Gaussian Processes with Sparse Data-Dependent Eigenfunctions

Sparse Orthogonal Variational Inference for Gaussian Processes

Compactly-supported nonstationary kernels for computing exact Gaussian processes on big data

Scalable GAM using sparse variational Gaussian processes