Abstract:A least squares support vector machine (LS-SVM) offers performance comparable to that of SVMs for classification and regression. The main limitation of LS-SVM is that it lacks sparsity compared with SVMs, making LS-SVM unsuitable for handling large-scale data due to computation and memory costs. To obtain sparse LS-SVM, several pruning methods based on an iterative strategy were recently proposed but did not consider the quantity constraint on the number of reserved support vectors, as widely used in real-life applications. In this article, a noniterative algorithm is proposed based on the selection of globally representative points (global-representation-based sparse least squares support vector machine, GRS-LSSVM) to improve the performance of sparse LS-SVM. For the first time, we present a model of sparse LS-SVM with a quantity constraint. In solving the optimal solution of the model, we find that using globally representative points to construct the reserved support vector set produces a better solution than other methods. We design an indicator based on point density and point dispersion to evaluate the global representation of points in feature space. Using the indicator, the top globally representative points are selected in one step from all points to construct the reserved support vector set of sparse LS-SVM. After obtaining the set, the decision hyperplane of sparse LS-SVM is directly computed using an algebraic formula. This algorithm only consumes O(N2) in computational complexity and O(N) in memory cost which makes it suitable for large-scale data sets. The experimental results show that the proposed algorithm has higher sparsity, greater stability, and lower computational complexity than the traditional iterative algorithms.

Joint Sample and Feature Selection Via Sparse Primal and Dual LSSVM

L1-Norm Least Squares Support Vector Regression Via the Alternating Direction Method of Multipliers

Sparse Lq-norm Least Squares Support Vector Machine with Feature Selection

Sparse Support Vector Machine with L P Penalty for Feature Selection

Feature selection for high‐dimensional regression via sparse LSSVR based on Lp‐norm

Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets.

Least Squares Support Vector Machine Regression Based on Sparse Samples and Mixture Kernel Learning.

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

Efficient Sparse Least Squares Support Vector Machines for Regression

Simultaneous Safe Feature and Sample Elimination for Sparse Support Vector Regression.

Noniterative Sparse LS-SVM Based on Globally Representative Point Selection

Least Squares Support Vector Machine with Self-Organizing Multiple Kernel Learning and Sparsity.

Discriminative Feature Selection Via A Structured Sparse Subspace Learning Module

Improved fuzzy sparse multi-class least squares support vector machine

Weighted Sparse Partial Least Squares for Joint Sample and Feature Selection

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction

Subspace Sparse Discriminative Feature Selection

Discriminative Sparse Subspace Learning and Its Application to Unsupervised Feature Selection

A direct sparse algorithm for LS-SVR

Automatic Spatial–Spectral Feature Selection for Hyperspectral Image Via Discriminative Sparse Multimodal Learning