Abstract:The Gaussian process (GP) is a widely used probabilistic machine learning method for stochastic function approximation, stochastic modeling, and analyzing real-world measurements of nonlinear processes. Unlike many other machine learning methods, GPs include an implicit characterization of uncertainty, making them extremely useful across many areas of science, technology, and engineering. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility and exact methods for inference that prevent application to data sets with more than about ten thousand points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space-time prediction based on more than one million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.

State Space representation of non-stationary Gaussian Processes

Identification of Gaussian Process State Space Models

Markovian Gaussian Process: A Universal State-Space Representation for Stationary Temporal Gaussian Process

Active data based Gaussian process models for nonlinear spatiotemporal systems

Towards Efficient Modeling and Inference in Multi-Dimensional Gaussian Process State-Space Models

State-space representation of Matérn and Damped Simple Harmonic Oscillator Gaussian processes

State Space Gaussian Process Prediction

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces

Compactly-supported nonstationary kernels for computing exact Gaussian processes on big data

State Space Models for Gaussian Stochastic Processes.

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the compact case

A Unifying Perspective on Non-Stationary Kernels for Deeper Gaussian Processes

Non-stationary Gaussian Process Surrogates

Physics-Informed Variational State-Space Gaussian Processes

Mixed-Stationary Gaussian Process for Flexible Non-Stationary Modeling of Spatial Outcomes

Online Gaussian Process State-Space Model: Learning and Planning for Partially Observable Dynamical Systems

Infinite-Horizon Gaussian Processes

Recursive Gaussian Process State Space Model

Separable Gaussian Neural Networks for High-Dimensional Nonlinear Stochastic Systems

Non-separable Spatio-temporal Graph Kernels via SPDEs

Efficient spatio-temporal Gaussian regression via Kalman filtering