Abstract:Support vector machines (SVMs) are invaluable tools for many practical applications in artificial intelligence, e.g., classification and event recognition. However, popular SVM solvers are not sufficiently efficient for applications with a great deal of samples as well as a large number of features. In this paper, thus, we present NESVM, a fast gradient SVM solver that can optimize various SVM models, e.g., classical SVM, linear programming SVM and least square SVM. Compared against SVM-Perf (whose convergence rate in solving the dual SVM is upper bounded by O(1/√k) where k is the number of iterations) and Pegasos (online SVM that converges at rate O(1/k) for the primal SVM), NESVM achieves the optimal convergence rate at O(1/k 2 ) and a linear time complexity. In particular, NESVM smoothes the nondifferentiable hinge loss and ℓ 1 -norm in the primal SVM. Then the optimal gradient method without any line search is adopted to solve the optimization. In each iteration round, the current gradient and historical gradients are combined to determine the descent direction, while the Lipschitz constant determines the step size. Only two matrix-vector multiplications are required in each iteration round. Therefore, NESVM is more efficient than existing SVM solvers. In addition, NESVM is available for both linear and nonlinear kernels. We also propose "homotopy NESVM" to accelerate NESVM by dynamically decreasing the smooth parameter and using the continuation method. Our experiments on census income categorization, indoor/outdoor scene classification event recognition and scene recognition suggest the efficiency and the effectiveness of NESVM. The MATLAB code of NESVM will be available on our website for further assessment.

Optimizing Top-k Multiclass SVM via Semismooth Newton Algorithm.

A Novel Svm Modeling Approach For Highly Imbalanced And Overlapping Classification

Top-k Multiclass SVM

Robust Top-k Multiclass SVM for Visual Category Recognition

A Simplified Multi-Class Support Vector Machine with Reduced Dual Optimization

NESVM: A Fast Gradient Method for Support Vector Machines

Enhancing Support Vector Machine Performance: A Hybrid Approach with Davidon-Fletcher-Powell Algorithm and Elephant Herding Optimization (EHO-DFP) for Parameter Optimization

Single Sequential Minimal Optimization: an Improved SVMs Training Algorithm

Least squares KNN-based weighted multiclass twin SVM

Hyper-parameter tuning for support vector machine using an improved cat swarm optimization algorithm

Non-smooth optimization algorithm to solve the LINEX soft support vector machine

A Semismooth-Newton's-Method-Based Linearization and Approximation Approach for Kernel Support Vector Machines

A fast smoothing newton method for bilevel hyperparameter optimization for SVC with logistic loss

A Robust Optimization Model for Nonlinear Support Vector Machine

Towards Optimal Discriminating Order for Multiclass Classification

Optimizing Kernel Transformations to Handle Binary Class Imbalanced Dataset Classification

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Two-stage sparse multi-kernel optimization classifier method for more accurate and explainable prediction

Global Optimization for Cardinality-constrained Minimum Sum-of-Squares Clustering via Semidefinite Programming

Parameter Optimization For Svm Using Sequential Number Theoretic For Optimization

A New Method for SVM Hyper-parameters Optimization