Abstract:Due to their flexibility and theoretical tractability Gaussian process (GP) regression models have become a central topic in modern statistics and machine learning. While the true posterior in these models is given explicitly, numerical evaluations depend on the inversion of the augmented kernel matrix $ K + \sigma^2 I $, which requires up to $ O(n^3) $ operations. For large sample sizes n, which are typically given in modern applications, this is computationally infeasible and necessitates the use of an approximate version of the posterior. Although such methods are widely used in practice, they typically have very limtied theoretical underpinning. In this context, we analyze a class of recently proposed approximation algorithms from the field of Probabilistic numerics. They can be interpreted in terms of Lanczos approximate eigenvectors of the kernel matrix or a conjugate gradient approximation of the posterior mean, which are particularly advantageous in truly large scale applications, as they are fundamentally only based on matrix vector multiplications amenable to the GPU acceleration of modern software frameworks. We combine result from the numerical analysis literature with state of the art concentration results for spectra of kernel matrices to obtain minimax contraction rates. Our theoretical findings are illustrated by numerical experiments.

Approximate Implementation of Logarithm of Matrix Determinant in Gaussian Process Regression and Numerical Experiments

Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression

Log-det approximation based on uniformly distributed seeds and its application to Gaussian process regression

Uniformly Distributed Seeds For Randomized Trace Estimator On O(N-2)-Operation Log-Det Approximation In Gaussian Process Regression

Efficient Gaussian process based on BFGS updating and logdet approximation

Suboptimal subspace construction for log-determinant approximation

Sparse inversion for derivative of log determinant

Large-scale Log-determinant Computation through Stochastic Chebyshev Expansions

Fast Approximate Determinants Using Rational Functions

Fast and Accurate Log-Determinant Approximations

High Dimensional Covariance Matrix Estimation by Penalizing the Matrix-Logarithm Transformed Likelihood

Contraction rates for conjugate gradient and Lanczos approximate posteriors in Gaussian process regression

Scaling Gaussian Process Regression with Derivatives

Polynomial approximations for the matrix logarithm with computation graphs

A preconditioning technique of Gauss--Legendre quadrature for the logarithm of symmetric positive definite matrices

Theoretical error estimates for computing the matrix logarithm by Padé-type approximants

Optimized Computation for Determinant of Multivariate Polynomial Matrices on GPGPU

Graph Estimation for Matrix-variate Gaussian Data

Fast Projected Newton-like Method for Precision Matrix Estimation under Total Positivity

Semidefinite approximations of the matrix logarithm