Abstract:Support vector machines (SVMs) are successful supervised learning models that analyze data for classification and regression. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the increase of the sample size brings great challenges to accurately and efficiently solve the large-scale SVMs, especially for the nonlinear kernel SVMs, which may lead to huge computational costs and unaffordable storage burden. In this paper, we propose a highly efficient sparse semismooth Newton (SsN) based augmented Lagrangian (AL) method for solving a class of large-scale SVMs that can be formulated as a convex quadratic programming problem with a linear equality constraint and a simple box constraint. The asymptotic superlinear convergence rate of both the primal and the dual iteration sequences generated by the AL method is guaranteed due to the piecewise linear-quadratic structure of the problem. Furthermore, we reveal the close connection between the number of support vectors and the sparse structure of the generalized Jacobian for the inner subproblem of the AL method. By exploiting this hidden sparsity, the inner subproblem can be solved by the SsN method efficiently and accurately, which greatly reduces the storage burden and computational costs. In particular, for the nonlinear kernel SVMs, since the sparse structure may not manifest in the early iterations of the AL method, we solve a linear kernel SVM approximated by the random Fourier features method to produce a good initial point, and then transfer to solve the original problem. Numerical experiments demonstrate that the proposed algorithm outperforms the current state-of-the-art solvers for the large-scale SVMs.

Global sensitivity analysis using Support Vector Regression

Novel Linear Search for Support Vector Machine Parameter Selection

Support Vector Regression-Based Active Subspace (SVR-AS) Modeling of High-Speed Links for Fast and Accurate Sensitivity Analysis

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Surrogate-assisted global sensitivity analysis: an overview

Global Nonlinear Kernel Prediction for Large Data Set With a Particle Swarm-Optimized Interval Support Vector Regression

Multiscale Support Vector Regression Method in Sobolev Spaces on Bounded Domains

Multikernel semiparametric linear programming support vector regression

A Fast Global Optimizer Based on Improved CS-RBF and Stochastic Optimal Algorithm

Application of Support Vector Machines to Accelerate the Solution Speed of Metaheuristic Algorithms

Study on geometric approach of SVM algorithm-SK algorithm analysis and study

Sensitivity Analysis for Computationally Expensive Models using Optimization and Objective-oriented Surrogate Approximations

Online adaptive machine learning based algorithm for implied volatility surface modeling

A screening approach for non-parametric global sensitivity analysis

Handling the impact of feature uncertainties on SVM: A robust approach based on Sobol sensitivity analysis

Data-driven Robust Optimization Based on Kernel Learning.

An Efficient Algorithm for a Class of Large-Scale Support Vector Machines Exploiting Hidden Sparsity.

Linear Programming-Based Sparse Kernel Regression with L1-Norm Minimization for Nonlinear System Modeling

SVR Kernel Parameters Selection Based on Steady-State Genetic Algorithm

Robust Support Vector Regression in the Primal

A support vector machine classifier based on a new kernel function model for hyperspectral data