Abstract:This paper presents a new approach for Gaussian process (GP) regression for large datasets. The approach involves partitioning the regression input domain into multiple local regions with a different local GP model fitted in each region. Unlike existing local partitioned GP approaches, we introduce a technique for patching together the local GP models nearly seamlessly to ensure that the local GP models for two neighboring regions produce nearly the same response prediction and prediction error variance on the boundary between the two regions. This largely mitigates the well-known discontinuity problem that degrades the boundary accuracy of existing local partitioned GP methods. Our main innovation is to represent the continuity conditions as additional pseudo-observations that the differences between neighboring GP responses are identically zero at an appropriately chosen set of boundary input locations. To predict the response at any input location, we simply augment the actual response observations with the pseudo-observations and apply standard GP prediction methods to the augmented data. In contrast to heuristic continuity adjustments, this has an advantage of working within a formal GP framework, so that the GP-based predictive uncertainty quantification remains valid. Our approach also inherits a sparse block-like structure for the sample covariance matrix, which results in computationally efficient closed-form expressions for the predictive mean and variance. In addition, we provide a new spatial partitioning scheme based on a recursive space partitioning along local principal component directions, which makes the proposed approach applicable for regression domains having more than two dimensions. Using three spatial datasets and three higher dimensional datasets, we investigate the numerical performance of the approach and compare it to several state-of-the-art approaches.

smashGP: Large-Scale Spatial Modeling via Matrix-Free Gaussian Processes

Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets

Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data

GP-HMAT: Scalable, ${O}(n\log(n))$ Gaussian Process Regression with Hierarchical Low-Rank Matrices

Large-Scale Gaussian Processes via Alternating Projection

Patchwork Kriging for Large-scale Gaussian Process Regression

ProSpar-GP: scalable Gaussian process modeling with massive non-stationary datasets

Towards Scalable Gaussian Process Modeling

Thoughts on Massively Scalable Gaussian Processes

Block Vecchia Approximation for Scalable and Efficient Gaussian Process Computations

A Case Study Competition Among Methods for Analyzing Large Spatial Data

Grid-Parametrize-Split (GriPS) for Improved Scalable Inference in Spatial Big Data Analysis

When Gaussian Process Meets Big Data: A Review of Scalable GPs

Scalable Gaussian Process Using Inexact Admm For Big Data

Highly Scalable Bayesian Geostatistical Modeling via Meshed Gaussian Processes on Partitioned Domains

A Robust Approach to Gaussian Processes Implementation

Representing Additive Gaussian Processes by Sparse Matrices

Compactly-supported nonstationary kernels for computing exact Gaussian processes on big data

Composite Inference for Gaussian Processes

High Performance Multivariate Geospatial Statistics on Manycore Systems

Solving High Frequency and Multi-Scale PDEs with Gaussian Processes