Abstract:Over the past two decades, support vector machine (SVM) has become a popular supervised machine learning model, and plenty of distinct algorithms are designed separately based on different KKT conditions of the SVM model for classification/regression with different losses, including the convex loss or nonconvex loss. In this paper, we propose an algorithm that can train different SVM models in a \emph{unified} scheme. First, we introduce a definition of the \emph{LS-DC} (\textbf{l}east \textbf{s}quares type of \textbf{d}ifference of \textbf{c}onvex) loss and show that the most commonly used losses in the SVM community are LS-DC loss or can be approximated by LS-DC loss. Based on DCA (difference of convex algorithm), we then propose a unified algorithm, called \emph{UniSVM}, which can solve the SVM model with any convex or nonconvex LS-DC loss, in which only a vector is computed, especially by the specifically chosen loss. Particularly, for training robust SVM models with nonconvex losses, UniSVM has a dominant advantage over all existing algorithms because it has a closed-form solution per iteration, while the existing algorithms always need to solve an L1SVM/L2SVM per iteration. Furthermore, by the low-rank approximation of the kernel matrix, UniSVM can solve the large-scale nonlinear problems with efficiency. To verify the efficacy and feasibility of the proposed algorithm, we perform many experiments on some small artificial problems and some large benchmark tasks with/without outliers for classification and regression for comparison with state-of-the-art algorithms. The experimental results demonstrate that UniSVM can achieve comparable performance in less training time. The foremost advantage of UniSVM is that its core code in Matlab is less than 10 lines; hence, it can be easily grasped by users or researchers.

Nonparallel Support Vector Machine with L2-norm Loss and Its DCD-type Solver

A Novel Robust Smooth Support Vector Machine

Training support vector data descriptors using converging linear particle swarm optimization

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

An On-Line Learning Approach with Support Vector Dormain Classifier

ν-Nonparallel support vector machine for pattern classification

NPSVC++: Nonparallel Classifiers Encounter Representation Learning

Rough margin based core vector machine

Unified SVM Algorithm Based on LS-DC Loss

A nonlinear kernel SVM classifier via L 0 / 1 soft-margin loss with classification performance

Non-smooth optimization algorithm to solve the LINEX soft support vector machine

CTSVM: A robust twin support vector machine with correntropy-induced loss function for binary classification problems

Coordinate Descent Algorithm for Ramp Loss Linear Programming Support Vector Machines

Smooth pinball loss nonparallel support vector machine for robust classification

Volumetric Error Prediction and Compensation of NC Machine Tool Based on Least Square Support Vector Machine

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Improving Sparsity and Scalability in Regularized Nonconvex Truncated-Loss Learning Problems

New method for solving Ivanov regularization-based support vector machine learning

A Feature Selection Method for Nonparallel Plane Support Vector Machine Classification

Cost-Sensitive Large Margin Distribution Machine

NESVM: A Fast Gradient Method for Support Vector Machines