Abstract:This paper presents a variational Bayesian kernel selection (VBKS) algorithm for sparse Gaussian process regression (SGPR) models. In contrast to existing GP kernel selection algorithms that aim to select only one kernel with the highest model evidence, our proposed VBKS algorithm considers the kernel as a random variable and learns its belief from data such that the uncertainty of the kernel can be interpreted and exploited to avoid overconfident GP predictions. To achieve this, we represent the probabilistic kernel as an additional variational variable in a variational inference (VI) framework for SGPR models where its posterior belief is learned together with that of the other variational variables (i.e., inducing variables and kernel hyperparameters). In particular, we transform the discrete kernel belief into a continuous parametric distribution via reparameterization in order to apply VI. Though it is computationally challenging to jointly optimize a large number of hyperparameters due to many kernels being evaluated simultaneously by our VBKS algorithm, we show that the variational lower bound of the log-marginal likelihood can be decomposed into an additive form such that each additive term depends only on a disjoint subset of the variational variables and can thus be optimized independently. Stochastic optimization is then used to maximize the variational lower bound by iteratively improving the variational approximation of the exact posterior belief via stochastic gradient ascent, which incurs constant time per iteration and hence scales to big data. We empirically evaluate the performance of our VBKS algorithm on synthetic and massive real-world datasets.

The Variant of Gaussian Kernel and Its Model Selection Method

SVM KERNEL PARAMETER SELECTION BASED ON IMPROVED GSO

Novel Linear Search for Support Vector Machine Parameter Selection

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

An Adaptive Gaussian Kernel for Support Vector Machine

Parameter selection of Gaussian kernel for one-class SVM.

A New Method for Determining the Parameter of Gaussian Kernel

A Novel Kernel-Based Maximum a Posteriori Classification Method

An information-geometrical method for improving the performance of support vector machine classifiers

A new trigonometric kernel function for support vector machine

Two Methods of Selecting Gaussian Kernel Parameters for One-Class Svm and Their Application to Fault Detection

Improving Support Vector Machine Classifiers By Modifying Kernel Functions

Behavioral analysis of support vector machine classifier with Gaussian kernel and imbalanced data

An Approach to Choosing Gaussian Kernel Parameter for One-Class SVMs Via Tightness Detecting

A New Class of Bessel Kernel Functions for Support Vector Machine

Gene Selection Using Gaussian Kernel Support Vector Machine Based Recursive Feature Elimination with Adaptive Kernel Width Strategy

A support vector machine classifier based on a new kernel function model for hyperspectral data

Scaling the kernel function to improve performance of the support vector machine

Feature Selection in a Kernel Space

Model Selection of Gaussian Kernel PCA for Novelty Detection

Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression