Abstract:Gaussian processes are a natural way of defining prior distributions over functions of one or more input variables. In a simple nonparametric regression problem, where such a function gives the mean of a Gaussian distribution for an observed response, a Gaussian process model can easily be implemented using matrix computations that are feasible for datasets of up to about a thousand cases. Hyperparameters that define the covariance function of the Gaussian process can be sampled using Markov chain methods. Regression models where the noise has a t distribution and logistic or probit models for classification applications can be implemented by sampling as well for latent values underlying the observations. Software is now available that implements these methods using covariance functions with hierarchical parameterizations. Models defined in this way can discover high-level properties of the data, such as which inputs are relevant to predicting the response.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to develop and implement the application of Gaussian process models in Bayesian regression and classification. Specifically, the paper focuses on the following aspects: 1. **Non - parametric regression models**: The paper explores how to use Gaussian processes as prior distributions to define non - parametric regression models on the function space. Gaussian processes can naturally represent the relationship between input variables and output variables and can be efficiently implemented through matrix calculations, which are suitable for problems with data set sizes within a few thousand samples. 2. **Estimation of hyper - parameters**: The paper discusses how to sample the hyper - parameters defining the covariance function of Gaussian processes through the Markov Chain Monte Carlo (MCMC) method. These hyper - parameters control the characteristics of the model, such as the noise level, the correlation of input variables, etc. 3. **Non - Gaussian noise and classification problems**: For regression models with non - Gaussian noise and classification problems, the paper proposes a method to handle these problems by introducing latent values. Latent values are used to define the target distribution of each observation, enabling the model to handle more complex data types, such as binary classification or multi - class classification problems. 4. **Software implementation**: The paper introduces a software package based on the C language that implements the regression and classification functions of the above - mentioned Gaussian process models. The software supports multiple covariance functions and can perform full Bayesian inference through the MCMC method. 5. **Model performance**: The paper shows that the prediction performance of the Gaussian process model is better than other non - parametric regression methods on multiple tasks. In addition, the concept of the Gaussian process model is simple and highly flexible, making it suitable for various problems. In summary, this paper aims to provide an efficient and flexible Gaussian process model framework for solving regression and classification problems and verifies its effectiveness and superiority through practical cases and experiments.

Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification

Learning Gaussian Networks

Minibatch Markov chain Monte Carlo Algorithms for Fitting Gaussian Processes

Robust and Conjugate Gaussian Process Regression

Latent Gaussian Process Regression

Robust Gaussian Process Regression with Huber Likelihood

Polynomial Representation of the Gaussian Process

GaussianProcesses.jl: A Nonparametric Bayes package for the Julia Language

Bayesian Modeling with Gaussian Processes using the GPstuff Toolbox

On Integrating Prior Knowledge into Gaussian Processes for Prognostic Health Monitoring

Fast and robust Bayesian inference using Gaussian processes with GPry

Towards Scalable Gaussian Process Modeling

Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes

Deep Horseshoe Gaussian Processes

Multi-Response Heteroscedastic Gaussian Process Models and Their Inference

Scalable Fully Bayesian Gaussian Process Modeling and Calibration With Adaptive Sequential Monte Carlo for Industrial Applications

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent

Bayesian Non-linear Latent Variable Modeling via Random Fourier Features

A Bayesian Take on Gaussian Process Networks

Gaussian Process Regression in the Flat Limit

Infinite dimensional adaptive MCMC for Gaussian processes