Abstract:This paper presents a variational Bayesian kernel selection (VBKS) algorithm for sparse Gaussian process regression (SGPR) models. In contrast to existing GP kernel selection algorithms that aim to select only one kernel with the highest model evidence, our proposed VBKS algorithm considers the kernel as a random variable and learns its belief from data such that the uncertainty of the kernel can be interpreted and exploited to avoid overconfident GP predictions. To achieve this, we represent the probabilistic kernel as an additional variational variable in a variational inference (VI) framework for SGPR models where its posterior belief is learned together with that of the other variational variables (i.e., inducing variables and kernel hyperparameters). In particular, we transform the discrete kernel belief into a continuous parametric distribution via reparameterization in order to apply VI. Though it is computationally challenging to jointly optimize a large number of hyperparameters due to many kernels being evaluated simultaneously by our VBKS algorithm, we show that the variational lower bound of the log-marginal likelihood can be decomposed into an additive form such that each additive term depends only on a disjoint subset of the variational variables and can thus be optimized independently. Stochastic optimization is then used to maximize the variational lower bound by iteratively improving the variational approximation of the exact posterior belief via stochastic gradient ascent, which incurs constant time per iteration and hence scales to big data. We empirically evaluate the performance of our VBKS algorithm on synthetic and massive real-world datasets.

SVR Kernel Parameters Selection Based on Steady-State Genetic Algorithm

SVM KERNEL PARAMETER SELECTION BASED ON IMPROVED GSO

Hyper Parameters Selection For Svr Based On Steady-State Genetic Algorithm

Novel Linear Search for Support Vector Machine Parameter Selection

Feature Selection and Parameter Optimization for Support Vector Machines: A New Approach Based on Genetic Algorithm with Feature Chromosomes.

The Optimization for Parameter of Kernel Function with GA

A Least Square Support Vector Machine Approach Based on Bvrna-Ga for Modeling Photovoltaic Systems

Swarm Intelligent Tuning Of One-Class V-Svm Parameters

Gene Selection Using Gaussian Kernel Support Vector Machine Based Recursive Feature Elimination with Adaptive Kernel Width Strategy

Parameter estimation for time-varying system based on improved genetic algorithm

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

Parameter choosing method of support vector regression for time series prediction

Multi-objective parameters selection for SVM classification using NSGA-II

Variable Selection in Logistic Regression Model with Genetic Algorithm.

Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression

Automatic optimized support vector regression for financial data prediction

Improving Genetic Algorithm Based Parameter Optimizing Lssvm Model for Tyre Regression

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Evolving Support Vector Machine Parameters

Support Vector Regression Parameters Optimization using Golden Sine Algorithm and its application in stock market

SVM Parameter Optimization using Grid Search and Genetic Algorithm to Improve Classification Performance