Abstract:A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Loève (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection thus becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. Theoretical computational complexities are in training and per point in inference, where N is the number of instances and P the number of expansion terms. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a 'Susceptible, Infected, Recovered' (SIR) toy problem, along with the experimental 'Cascaded Tanks' benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.

Active data based Gaussian process models for nonlinear spatiotemporal systems

Self-active and Recursively Selective Gaussian Process Models for Nonlinear Distributed Parameter Systems

Nonparametric Identification Based on Gaussian Process Regression for Distributed Parameter Systems

Active Design of Dynamic GP Models for Model Predictive Control Using Expected Improvement

Efficient spatio-temporal Gaussian regression via Kalman filtering

A spatial Gaussian process method for hyperspectral remote sensing imagery classification

Identification of Gaussian Process State Space Models

Numerical Gaussian Process Kalman Filtering for Spatiotemporal Systems

Mixed-Stationary Gaussian Process for Flexible Non-Stationary Modeling of Spatial Outcomes

State Estimation for Nonlinear Dynamic Systems Using Gaussian Processes and Pre-Computed Local Linear Models

Physics-Informed Variational State-Space Gaussian Processes

Compactly-supported nonstationary kernels for computing exact Gaussian processes on big data

Nonparametric identification of batch process using two-dimensional kernel-based Gaussian process regression

Markovian Gaussian Process: A Universal State-Space Representation for Stationary Temporal Gaussian Process

Gaussian Processes for Analyzing Positioned Trajectories in Sports

Learning Non-Stationary Space-Time Models for Environmental Monitoring

Latent Gaussian Process Regression

ProSpar-GP: scalable Gaussian process modeling with massive non-stationary datasets

EigenGP: Gaussian Processes with Sparse Data-Dependent Eigenfunctions

Fitting Spatial-Temporal Data Via a Physics Regularized Multi-Output Grid Gaussian Process: Case Studies of a Bike-Sharing System

Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed Gaussian processes