Abstract:In this paper, a sparse learning algorithm, probabilistic classification vector machines (PCVMs), is proposed. We analyze relevance vector machines (RVMs) for classification problems and observe that adopting the same prior for different classes may lead to unstable solutions. In order to tackle this problem, a signed and truncated Gaussian prior is adopted over every weight in PCVMs, where the sign of prior is determined by the class label, i.e., +1 or -1. The truncated Gaussian prior not only restricts the sign of weights but also leads to a sparse estimation of weight vectors, and thus controls the complexity of the model. In PCVMs, the kernel parameters can be optimized simultaneously within the training algorithm. The performance of PCVMs is extensively evaluated on four synthetic data sets and 13 benchmark data sets using three performance metrics, error rate (ERR), area under the curve of receiver operating characteristic (AUC), and root mean squared error (RMSE). We compare PCVMs with soft-margin support vector machines (SVM(Soft)), hard-margin support vector machines (SVM(Hard)), SVM with the kernel parameters optimized by PCVMs (SVM(PCVM)), relevance vector machines (RVMs), and some other baseline classifiers. Through five replications of twofold cross-validation F test, i.e., 5 x 2 cross-validation F test, over single data sets and Friedman test with the corresponding post-hoc test to compare these algorithms over multiple data sets, we notice that PCVMs outperform other algorithms, including SVM(Soft), SVM(Hard), RVM, and SVM(PCVM), on most of the data sets under the three metrics, especially under AUC. Our results also reveal that the performance of SVM(PCVM) is slightly better than SVM(Soft), implying that the parameter optimization algorithm in PCVMs is better than cross validation in terms of performance and computational complexity. In this paper, we also discuss the superiority of PCVMs' formulation using maximum a posteriori (MAP) analysis and margin analysis, which explain the empirical success of PCVMs.

Estimating the Generalization Performance of Polynomial SVM Classifier for Text Categorization

SVM Classification:Its Contents and Challenges

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

Feature Selection for Support Vector Machines in Text Categorization

Generalization Performance Analysis of M-SVMs

On Improvement on Generalization Performance of Classifier by Using Empirical Risk

Dimensionality Reduction With Category Information Fusion And Non-Negative Matrix Factorization For Text Categorization

Application of SVM in web page categorization.

Web Page Classification Based on SVM

Efficient Text Categorization Using A Min-Max Modular Support Vector Machine

A Grouped Structure-based Regularized Regression Model for Text Categorization

Fast Text Categorization with Min-Max Modular Support Vector Machines

A class-feature-centroid classifier for text categorization

Explanation based generalized /spl epsi/-SVM and its application in intelligent project management

Explanation Based Generalized (Epsilon)-Svm And Its Application In Intelligent Project Management

Study on SVM Compared with the other Text Classification Methods

An Adaptive Error Penalization Method for Training an Efficient and Generalized SVM.

Appropriateness in Applying SVMs to Text Classification

Estimation of the Misclassification Error for Multicategory Support Vector Machine Classification

Probabilistic Classification Vector Machines.

Rough sets method for SVM data preprocessing