Abstract:Classification and probability estimation have broad applications in modern machine learning and data science applications, including biology, medicine, engineering, and computer science. The recent development of a class of weighted Support Vector Machines (wSVMs) has shown great values in robustly predicting the class probability and classification for various problems with high accuracy. The current framework is based on the $\ell^2$-norm regularized binary wSVMs optimization problem, which only works with dense features and has poor performance at sparse features with redundant noise in most real applications. The sparse learning process requires a prescreen of the important variables for each binary wSVMs for accurately estimating pairwise conditional probability. In this paper, we proposed novel wSVMs frameworks that incorporate automatic variable selection with accurate probability estimation for sparse learning problems. We developed efficient algorithms for effective variable selection for solving either the $\ell^1$-norm or elastic net regularized binary wSVMs optimization problems. The binary class probability is then estimated either by the $\ell^2$-norm regularized wSVMs framework with selected variables or by elastic net regularized wSVMs directly. The two-step approach of $\ell^1$-norm followed by $\ell^2$-norm wSVMs show a great advantage in both automatic variable selection and reliable probability estimators with the most efficient time. The elastic net regularized wSVMs offer the best performance in terms of variable selection and probability estimation with the additional advantage of variable grouping in the compensation of more computation time for high dimensional problems. The proposed wSVMs-based sparse learning methods have wide applications and can be further extended to $K$-class problems through ensemble learning.

A Sparse Sampling Method for Classification Based on Likelihood Factor

Uncertainty-Based Active Learning Via Sparse Modeling for Image Classification

Imbalanced Data Classification Algorithm Based on Integrated Sampling and Ensemble Learning.

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction

Sparse Least Squares Support Vector Machine for Function Estimation

Sparse linear discriminant analysis by thresholding for high dimensional data

Sparse Learning and Class Probability Estimation with Weighted Support Vector Machines

Sample Weighting: an Inherent Approach for Outlier Suppressing Discriminant Analysis

A New Method of Sample Reduction for Support Vector Classification

A Fast and Robust Sparse Approach for Hyperspectral Data Classification Using a Few Labeled Samples

Improved fuzzy sparse multi-class least squares support vector machine

Dimension Reduction of Hyperspectral Images with Sparse Linear Discriminant Analysis

Hyperspectral Image Classification Via Sparse Code Histogram

Online Kernel Learning with a Near Optimal Sparsity Bound

Large-scale support vector machine classification with redundant data reduction

Probabilistic Feature Selection and Classification Vector Machine

A Classfication Method For Imbalance Data Set Based on Kernel SMOTE

Optimal Feature Selection for Sparse Linear Discriminant Analysis and Its Applications in Gene Expression Data

A New Sampling-Based Svm For Face Recognition

Discriminative feature extraction based on sparse and low-rank representation

Multi-resolution subsampling for large-scale linear classification