Abstract:In this paper, a sparse learning algorithm, probabilistic classification vector machines (PCVMs), is proposed. We analyze relevance vector machines (RVMs) for classification problems and observe that adopting the same prior for different classes may lead to unstable solutions. In order to tackle this problem, a signed and truncated Gaussian prior is adopted over every weight in PCVMs, where the sign of prior is determined by the class label, i.e., +1 or -1. The truncated Gaussian prior not only restricts the sign of weights but also leads to a sparse estimation of weight vectors, and thus controls the complexity of the model. In PCVMs, the kernel parameters can be optimized simultaneously within the training algorithm. The performance of PCVMs is extensively evaluated on four synthetic data sets and 13 benchmark data sets using three performance metrics, error rate (ERR), area under the curve of receiver operating characteristic (AUC), and root mean squared error (RMSE). We compare PCVMs with soft-margin support vector machines (SVM(Soft)), hard-margin support vector machines (SVM(Hard)), SVM with the kernel parameters optimized by PCVMs (SVM(PCVM)), relevance vector machines (RVMs), and some other baseline classifiers. Through five replications of twofold cross-validation F test, i.e., 5 x 2 cross-validation F test, over single data sets and Friedman test with the corresponding post-hoc test to compare these algorithms over multiple data sets, we notice that PCVMs outperform other algorithms, including SVM(Soft), SVM(Hard), RVM, and SVM(PCVM), on most of the data sets under the three metrics, especially under AUC. Our results also reveal that the performance of SVM(PCVM) is slightly better than SVM(Soft), implying that the parameter optimization algorithm in PCVMs is better than cross validation in terms of performance and computational complexity. In this paper, we also discuss the superiority of PCVMs' formulation using maximum a posteriori (MAP) analysis and margin analysis, which explain the empirical success of PCVMs.

A genetic multiple kernel relevance vector regression approach

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Support Vector Machines Ensemble With Optimizing Weights By Genetic Algorithm

A novel classification algorithm based on Morlet wavelet kernel relevance vector machine

Nonlinear Soft Sensor Development Based on Relevance Vector Machine

Probabilistic Classification Vector Machines.

An efficient multiple kernel computation method for regression analysis of economic data

Gene Selection Using Gaussian Kernel Support Vector Machine Based Recursive Feature Elimination with Adaptive Kernel Width Strategy

Multikernel semiparametric linear programming support vector regression

A joint introduction to Gaussian Processes and Relevance Vector Machines with connections to Kalman filtering and other kernel smoothers

Heteroscedastic Relevance Vector Machine

Parameters Selection in Gene Selection Using Gaussian Kernel Support Vector Machines by Genetic Algorithm

Research progress and analysis on methods for classification of RVM

Multiclass Kernel Polarization and Its Application to Parameter Selection of RBF Kernel with Multiple Widths

A Novel Kernel-Based Maximum a Posteriori Classification Method

Regression with Support Vector Machines and VGG Neural Networks.

Robust kernel ensemble regression in diversified kernel space with shared parameters

SVR Kernel Parameters Selection Based on Steady-State Genetic Algorithm

A Unified and Comprehensible View of Parametric and Kernel Methods for Genomic Prediction with Application to Rice

A Learning Algorithm with Gaussian Regularizer for Kernel Neuron

Multiple graph kernel learning based on GMDH-type neural network