Abstract:Support vector machines (SVMs) are successful supervised learning models that analyze data for classification and regression. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the increase of the sample size brings great challenges to accurately and efficiently solve the large-scale SVMs, especially for the nonlinear kernel SVMs, which may lead to huge computational costs and unaffordable storage burden. In this paper, we propose a highly efficient sparse semismooth Newton (SsN) based augmented Lagrangian (AL) method for solving a class of large-scale SVMs that can be formulated as a convex quadratic programming problem with a linear equality constraint and a simple box constraint. The asymptotic superlinear convergence rate of both the primal and the dual iteration sequences generated by the AL method is guaranteed due to the piecewise linear-quadratic structure of the problem. Furthermore, we reveal the close connection between the number of support vectors and the sparse structure of the generalized Jacobian for the inner subproblem of the AL method. By exploiting this hidden sparsity, the inner subproblem can be solved by the SsN method efficiently and accurately, which greatly reduces the storage burden and computational costs. In particular, for the nonlinear kernel SVMs, since the sparse structure may not manifest in the early iterations of the AL method, we solve a linear kernel SVM approximated by the random Fourier features method to produce a good initial point, and then transfer to solve the original problem. Numerical experiments demonstrate that the proposed algorithm outperforms the current state-of-the-art solvers for the large-scale SVMs.

Support vector machines within a bivariate mixed-integer linear programming framework

A Novel Svm Modeling Approach For Highly Imbalanced And Overlapping Classification

A mixed integer linear programming support vector machine for cost-effective feature selection

Support Feature Machines

Optimization meets Machine Learning: An Exact Algorithm for Semi-Supervised Support Vector Machines

Multikernel semiparametric linear programming support vector regression

A VSC Scheme for Linear MIMO Systems Based on SVM

Application of Support Vector Machines to Accelerate the Solution Speed of Metaheuristic Algorithms

A Robust Optimization Model for Nonlinear Support Vector Machine

New mixed integer fractional programming problem and some multi-objective models for sparse optimization

Mixed-integer linear optimization formulations for feature subset selection in kernel SVM classification

A Confident Majority Voting Strategy for Parallel and Modular Support Vector Machines

Feature subset selection for kernel SVM classification via mixed-integer optimization

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Machine Learning Augmented Branch and Bound for Mixed Integer Linear Programming

A Joint Learning Framework for Optimal Feature Extraction and Multi-class SVM

On $\ell_p$-Support Vector Machines and Multidimensional Kernels

A Multi-Agent Learning Framework for Mixed-Integer Linear Programming

High-Dimensional Penalized Bernstein Support Vector Machines

An Efficient Algorithm for a Class of Large-Scale Support Vector Machines Exploiting Hidden Sparsity.

Scaling Support Vector Machines on Modern HPC Platforms