Abstract:The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process regression (GPR), a well-known nonparametric, and interpretable Bayesian model, which suffers from cubic complexity to data size. To improve the scalability while retaining desirable prediction quality, a variety of scalable GPs have been presented. However, they have not yet been comprehensively reviewed and analyzed to be well understood by both academia and industry. The review of scalable GPs in the GP community is timely and important due to the explosion of data size. To this end, this article is devoted to reviewing state-of-the-art scalable GPs involving two main categories: global approximations that distillate the entire data and local approximations that divide the data for subspace learning. Particularly, for global approximations, we mainly focus on sparse approximations comprising prior approximations that modify the prior but perform exact inference, posterior approximations that retain exact prior but perform approximate inference, and structured sparse approximations that exploit specific structures in kernel matrix; for local approximations, we highlight the mixture/product of experts that conducts model averaging from multiple local experts to boost predictions. To present a complete review, recent advances for improving the scalability and capability of scalable GPs are reviewed. Finally, the extensions and open issues of scalable GPs in various scenarios are reviewed and discussed to inspire novel ideas for future research avenues.

Distributed Gaussian Process: New Paradigm and Application to Wireless Traffic Prediction.

Deep Graph Gaussian Processes for Short-Term Traffic Flow Forecasting from Spatiotemporal Data

A Bayesian clustering ensemble Gaussian process model for network-wide traffic flow clustering and prediction

A Spatial-Temporal Gated Hypergraph Convolution Network for Traffic Prediction

AGNP: Network-wide Short-Term Probabilistic Traffic Speed Prediction and Imputation

Pattern Discovery and Multi-Slot-Ahead Forecast of Network Traffic: A Revisiting to Gaussian Process

Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey

Asynchronous Distributed Variational Gaussian Process for Regression.

High-Order Gaussian Process Dynamical Models for Traffic Flow Prediction

Latent Gaussian Processes based Graph Learning for Urban Traffic Prediction

Gaussian Processes for Traffic Speed Prediction at Different Aggregation Levels

Understanding and comparing scalable Gaussian process regression for big data

Aggregation Models with Optimal Weights for Distributed Gaussian Processes

Whom to Trust? Elective Learning for Distributed Gaussian Process Regression

A Gaussian-Process-Based Data-Driven Traffic Flow Model and Its Application in Road Capacity Analysis

Multi-Output Gaussian Processes for Crowdsourced Traffic Data Imputation

Optimal Composite Likelihood Estimation and Prediction for Distributed Gaussian Process Modeling.

When Gaussian Process Meets Big Data: A Review of Scalable GPs

Spatiotemporal Gaussian Process Kalman Filter for Mobile Traffic Prediction.

Fitting Spatial-Temporal Data Via a Physics Regularized Multi-Output Grid Gaussian Process: Case Studies of a Bike-Sharing System

Gaussian Process Regression with Grid Spectral Mixture Kernel: Distributed Learning for Multidimensional Data