Abstract:Classification and probability estimation have broad applications in modern machine learning and data science applications, including biology, medicine, engineering, and computer science. The recent development of a class of weighted Support Vector Machines (wSVMs) has shown great values in robustly predicting the class probability and classification for various problems with high accuracy. The current framework is based on the $\ell^2$-norm regularized binary wSVMs optimization problem, which only works with dense features and has poor performance at sparse features with redundant noise in most real applications. The sparse learning process requires a prescreen of the important variables for each binary wSVMs for accurately estimating pairwise conditional probability. In this paper, we proposed novel wSVMs frameworks that incorporate automatic variable selection with accurate probability estimation for sparse learning problems. We developed efficient algorithms for effective variable selection for solving either the $\ell^1$-norm or elastic net regularized binary wSVMs optimization problems. The binary class probability is then estimated either by the $\ell^2$-norm regularized wSVMs framework with selected variables or by elastic net regularized wSVMs directly. The two-step approach of $\ell^1$-norm followed by $\ell^2$-norm wSVMs show a great advantage in both automatic variable selection and reliable probability estimators with the most efficient time. The elastic net regularized wSVMs offer the best performance in terms of variable selection and probability estimation with the additional advantage of variable grouping in the compensation of more computation time for high dimensional problems. The proposed wSVMs-based sparse learning methods have wide applications and can be further extended to $K$-class problems through ensemble learning.

Weighted Least Squares Support Vector Machine for Semi-supervised Classification

Least Squares Support Vector Machines Based on Support Vector Degrees

Improved Weighted Least Squares Support Vector Machines Algorithm And Its Applications In Spectroscopic Quantitative Analysis

A Type-2 Fuzzy Logic Ensemble SVM Classifier Based on Feature Weighting

Semi-supervised Image Classification Via Nonnegative Least-Squares Regression

Improved Sparse Least-Squares Support Vector Machine Classifiers.

A Robust Pruning Method For Reduced Weighted Least Squares Support Vector Machine

A New Weighted Least Squares Support Vector Machines and Its Sequential Minimal Optimization Algorithm

Adaptive Semi-Supervised Learning with Discriminative Least Squares Regression

Density Weighted Pruning Method for Sparse Least Squares Support Vector Machines

Kernel density weighted method of obtaining the least square support vector machines weightes

Weighted Least Squares Projection Twin Support Vector Machines With Local Information

Sparse Learning and Class Probability Estimation with Weighted Support Vector Machines

A Performance Nondestructive Pruning Method For Reduced Robust Weighted Least Squares Support Vector Regression

Semi-supervised learning with extremely sparse labeled data on multiple semi-supervised assumptions

Improved fuzzy sparse multi-class least squares support vector machine

Discriminative and robust least squares regression for semi-supervised image classification

Semi-supervised spectral-spatial weighted classification of hyperspectral image based on SVMSLR framework

Study of on-line weighted least squares support vector machines

Least squares KNN-based weighted multiclass twin SVM

A Pruning Strategy Based On Confidence Interval For Sparse Ls-Svm