Abstract:A least squares support vector machine (LS-SVM) offers performance comparable to that of SVMs for classification and regression. The main limitation of LS-SVM is that it lacks sparsity compared with SVMs, making LS-SVM unsuitable for handling large-scale data due to computation and memory costs. To obtain sparse LS-SVM, several pruning methods based on an iterative strategy were recently proposed but did not consider the quantity constraint on the number of reserved support vectors, as widely used in real-life applications. In this article, a noniterative algorithm is proposed based on the selection of globally representative points (global-representation-based sparse least squares support vector machine, GRS-LSSVM) to improve the performance of sparse LS-SVM. For the first time, we present a model of sparse LS-SVM with a quantity constraint. In solving the optimal solution of the model, we find that using globally representative points to construct the reserved support vector set produces a better solution than other methods. We design an indicator based on point density and point dispersion to evaluate the global representation of points in feature space. Using the indicator, the top globally representative points are selected in one step from all points to construct the reserved support vector set of sparse LS-SVM. After obtaining the set, the decision hyperplane of sparse LS-SVM is directly computed using an algebraic formula. This algorithm only consumes O(N2) in computational complexity and O(N) in memory cost which makes it suitable for large-scale data sets. The experimental results show that the proposed algorithm has higher sparsity, greater stability, and lower computational complexity than the traditional iterative algorithms.

An Efficient Algorithm for a Class of Large-Scale Support Vector Machines Exploiting Hidden Sparsity.

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Sparse Least Squares Support Vector Machine for Function Estimation

Sparse and Robust SVM Classifier for Large Scale Classification

Noniterative Sparse LS-SVM Based on Globally Representative Point Selection

Stochastic Sequential Minimal Optimization For Large-Scale Linear Svm

Large-scale Linear Nonparallel SVMs

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines.

More Efficient Sparse Multi-kernel Based Least Square Support Vector Machine

Efficient Sparse Least Squares Support Vector Machines for Regression

Linear Regression-Based Efficient SVM Learning for Large-Scale Classification.

Support matrix machine: exploring sample sparsity, low rank, and adaptive sieving in high-performance computing

A two stages sparse SVM training

Robust support vector machines and their sparse algorithms

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction

Semismooth Newton Support Vector Machine

A fast SVM training method based on sparse hyperplane-fitting for large scale problems

Fast Sparse Approximation for Least Squares Support Vector Machine.

A distributed sequential solver for large-scale svms

Fast Algorithms for Linear and Kernel SVM+.