Abstract:Kriging and Gaussian Process Regression are statistical methods that allow predicting the outcome of a random process or a random field by using a sample of correlated observations. In other words, the random process or random field is partially observed, and by using a sample a prediction is made, pointwise or as a whole, where the latter can be thought as a reconstruction. In addition, the techniques permit to give a measure of uncertainty of the prediction. The methods have different origins. Kriging comes from geostatistics, a field which started to develop around 1950 oriented to mining valuation problems, whereas Gaussian Process Regression has gained popularity in the area of machine learning in the last decade of the previous century. In the literature, the methods are usually presented as being the same technique. However, beyond this affirmation, the techniques have yet not been compared on a thorough mathematical basis and neither explained why and under which conditions this affirmation holds. Furthermore, Kriging has many variants and this affirmation should be precised. In this paper, this gap is filled. It is shown, step by step how both methods are deduced from the first principles -- with a major focus on Kriging, the mathematical connection between them, and which Kriging variant corresponds to which Gaussian Process Regression set up. The three most widely used versions of Kriging are considered: Simple Kriging, Ordinary Kriging and Universal Kriging. It is found, that despite their closeness, the techniques are different in their approach and assumptions, in a similar way the Least Square method, the Best Linear Unbiased Estimator method, and the Likelihood method in regression do. I hope this work can serve for a deeper understanding of the relationship between Kriging and Gaussian Process Regression, as well as a cohesive introductory resource for researchers.

Universal Convergence of Kriging

On Prediction Properties of Kriging: Uniform Error Bounds and Robustness

Kriging prediction with isotropic Matérn correlations: Robustness and experimental design

Enhanced Universal Kriging for Transformed Input Parameter Spaces

Including A Nugget Effect In Lifted Brownian Covariance Models

Explaining and Connecting Kriging with Gaussian Process Regression

A Comprehensive Bayesian Treatment of the Universal Kriging model with Matérn correlation kernels

Convergence of Gaussian process regression: Optimality, robustness, and relationship with kernel ridge regression

Universal kernel-type estimation of random fields

A Statistical Learning View of Simple Kriging

A novel general method for simulating a one dimensional random field based on the active learning Kriging model

Convergence Analysis of Stochastic Kriging-Assisted Simulation with Random Covariates

Extended Gaussian Kriging for computer experiments in engineering design

An adaptive kriging method for solving nonlinear inverse statistical problems

Non-Homogeneity Estimation and Universal Kriging on the Sphere

Multilevel approximation of Gaussian random fields: Covariance compression, estimation, and spatial prediction

On the Inference of Applying Gaussian Process Modeling to a Deterministic Function

A bayesian approach to kriging metamodeling for computer experiments

Efficient Prediction Designs for Random Fields

Convergence rates of non-stationary and deep Gaussian process regression

Patchwork Kriging for Large-scale Gaussian Process Regression