Abstract:In this paper, a sparse learning algorithm, probabilistic classification vector machines (PCVMs), is proposed. We analyze relevance vector machines (RVMs) for classification problems and observe that adopting the same prior for different classes may lead to unstable solutions. In order to tackle this problem, a signed and truncated Gaussian prior is adopted over every weight in PCVMs, where the sign of prior is determined by the class label, i.e., +1 or -1. The truncated Gaussian prior not only restricts the sign of weights but also leads to a sparse estimation of weight vectors, and thus controls the complexity of the model. In PCVMs, the kernel parameters can be optimized simultaneously within the training algorithm. The performance of PCVMs is extensively evaluated on four synthetic data sets and 13 benchmark data sets using three performance metrics, error rate (ERR), area under the curve of receiver operating characteristic (AUC), and root mean squared error (RMSE). We compare PCVMs with soft-margin support vector machines (SVM(Soft)), hard-margin support vector machines (SVM(Hard)), SVM with the kernel parameters optimized by PCVMs (SVM(PCVM)), relevance vector machines (RVMs), and some other baseline classifiers. Through five replications of twofold cross-validation F test, i.e., 5 x 2 cross-validation F test, over single data sets and Friedman test with the corresponding post-hoc test to compare these algorithms over multiple data sets, we notice that PCVMs outperform other algorithms, including SVM(Soft), SVM(Hard), RVM, and SVM(PCVM), on most of the data sets under the three metrics, especially under AUC. Our results also reveal that the performance of SVM(PCVM) is slightly better than SVM(Soft), implying that the parameter optimization algorithm in PCVMs is better than cross validation in terms of performance and computational complexity. In this paper, we also discuss the superiority of PCVMs' formulation using maximum a posteriori (MAP) analysis and margin analysis, which explain the empirical success of PCVMs.

The Study Of An Improved Relevant Vector Machine Mud Based On Perfect Sampling

M-estimator Based Support Vector Machine and Its Application

RVM-based classification of the network video surveillance system

SVM multiuser detection based on heuristic kernel

Probabilistic tangent subspace method for multiuser detection

Study of Multiuser Detection: the Support Vector Machine Approach.

Complexity Reduction of Signal Detection by Exploiting Correlation Characteristics of Spreading Sequences

Online Training of SVM for Multiuser Detection

A New Optimal MUD Algorithm Based on Searching in the Local Space of Sub-Optimal MMSE Solutions for DS-UWB Communication Systems

A Robust Blind Adaptive Multiuser Detection Based on Maximum Correntropy Criterion in Satellite CDMA Systems

Blind Multiuser Detection Based on Kernel Approximation

GMM-MUD: an Effective Multiuser Detection Algorithm for DS-UWB-Based Space Formation Flying Systems

Bayesian Learning Based Multiuser Detection for M2M Communications with Time-Varying User Activities

A novel blind multiuser detection algorithm based on support vector machines

Nonlinear Soft Sensor Development Based on Relevance Vector Machine

Relevance Vector Machine Based on Bayesian Learning and Its Application in Soft Sensing

Multiuser Detection with Particle Filtering

Investigation on the Construction of the Relevance Vector Machine Based on Cross Entropy Minimization

Probabilistic Classification Vector Machines.

Generalization Performance of Gaussian Kernels SVMC Based on Markov Sampling.

Improved Hyperspectral Land-Cover Analysis Using Relevance Vector Machine