Abstract:Classification and probability estimation have broad applications in modern machine learning and data science applications, including biology, medicine, engineering, and computer science. The recent development of a class of weighted Support Vector Machines (wSVMs) has shown great values in robustly predicting the class probability and classification for various problems with high accuracy. The current framework is based on the $\ell^2$-norm regularized binary wSVMs optimization problem, which only works with dense features and has poor performance at sparse features with redundant noise in most real applications. The sparse learning process requires a prescreen of the important variables for each binary wSVMs for accurately estimating pairwise conditional probability. In this paper, we proposed novel wSVMs frameworks that incorporate automatic variable selection with accurate probability estimation for sparse learning problems. We developed efficient algorithms for effective variable selection for solving either the $\ell^1$-norm or elastic net regularized binary wSVMs optimization problems. The binary class probability is then estimated either by the $\ell^2$-norm regularized wSVMs framework with selected variables or by elastic net regularized wSVMs directly. The two-step approach of $\ell^1$-norm followed by $\ell^2$-norm wSVMs show a great advantage in both automatic variable selection and reliable probability estimators with the most efficient time. The elastic net regularized wSVMs offer the best performance in terms of variable selection and probability estimation with the additional advantage of variable grouping in the compensation of more computation time for high dimensional problems. The proposed wSVMs-based sparse learning methods have wide applications and can be further extended to $K$-class problems through ensemble learning.

Weighted Lagrange Ε-Twin Support Vector Regression

Weighted Support Vector Machines for Regression and Its Application

Flatheaded support vector machine for regression

Robust Iterative Algorithm of Weighted Least Squares Support Vector Machine and Its Application in Spectral Analysis

L1-Norm Least Squares Support Vector Regression Via the Alternating Direction Method of Multipliers

Weighted Twin Support Vector Machines with Local Information and Its Application.

e-Distance Weighted Support Vector Regression

Robust Twin Support Vector Regression with Smooth Truncated Hε Loss Function

Twin support vector quantile regression

Symmetric LINEX loss twin support vector machine for robust classification and its fast iterative algorithm

On Regularization Based Twin Support Vector Regression with Huber Loss

CTSVM: A robust twin support vector machine with correntropy-induced loss function for binary classification problems

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Weighted least squares twin support vector machine based on density peaks

Sparse Learning and Class Probability Estimation with Weighted Support Vector Machines

Multi-hyperplane twin support vector regression guided with fuzzy clustering

Weighted LAD-LASSO method for robust parameter estimation and variable selection in regression

Comprehensive Review On Twin Support Vector Machines

Elastic Net Nonparallel Hyperplane Support Vector Machine and Its Geometrical Rationality

Weighted least squares regression with the best robustness and high computability

Enhancing Robustness and Efficiency of Least Square Twin SVM via Granular Computing