Abstract:Support vector machines (SVMs) are invaluable tools for many practical applications in artificial intelligence, e.g., classification and event recognition. However, popular SVM solvers are not sufficiently efficient for applications with a great deal of samples as well as a large number of features. In this paper, thus, we present NESVM, a fast gradient SVM solver that can optimize various SVM models, e.g., classical SVM, linear programming SVM and least square SVM. Compared against SVM-Perf (whose convergence rate in solving the dual SVM is upper bounded by O(1/√k) where k is the number of iterations) and Pegasos (online SVM that converges at rate O(1/k) for the primal SVM), NESVM achieves the optimal convergence rate at O(1/k 2 ) and a linear time complexity. In particular, NESVM smoothes the nondifferentiable hinge loss and ℓ 1 -norm in the primal SVM. Then the optimal gradient method without any line search is adopted to solve the optimization. In each iteration round, the current gradient and historical gradients are combined to determine the descent direction, while the Lipschitz constant determines the step size. Only two matrix-vector multiplications are required in each iteration round. Therefore, NESVM is more efficient than existing SVM solvers. In addition, NESVM is available for both linear and nonlinear kernels. We also propose "homotopy NESVM" to accelerate NESVM by dynamically decreasing the smooth parameter and using the continuation method. Our experiments on census income categorization, indoor/outdoor scene classification event recognition and scene recognition suggest the efficiency and the effectiveness of NESVM. The MATLAB code of NESVM will be available on our website for further assessment.

A Fast-Convergence Distributed Support Vector Machine in Small-Scale Strongly Connected Networks

A Parallel and Scalable Digital Architecture for Training Support Vector Machines

Distributed Online Semi-Supervised Support Vector Machine

Fast learning algorithm for support vector domain description

Distributed Online One-Class Support Vector Machine for Anomaly Detection over Networks

Parallel and Distributed Structured SVM Training

SignSGD with Federated Voting

Parallelizing Support Vector Machines on Distributed Computers

A Finite Time Discrete Distributed Learning Algorithm Using Stochastic Configuration Network

GADGET SVM: A Gossip-bAseD sub-GradiEnT Solver for Linear SVMs

On the Convergence of Communication-Efficient Local SGD for Federated Learning

Learning Performance of Weighted Distributed Learning With Support Vector Machines

Fast extraction strategy of support vector machines

Support vector machine in big data: smoothing strategy and adaptive distributed inference

NESVM: A Fast Gradient Method for Support Vector Machines

Faster Convergence with Less Communication: Broadcast-Based Subgraph Sampling for Decentralized Learning over Wireless Networks

Fast Training Support Vector Machines Using Parallel Sequential Minimal Optimization

An Online Incremental Learning Support Vector Machine for Large-Scale Data

Robust Fully-Asynchronous Methods for Distributed Training over General Architecture

D-SVM over Networked Systems with Non-Ideal Linking Conditions

A Fast Online SVM Algorithm for Multi-user Detection