Abstract:A least squares support vector machine (LS-SVM) offers performance comparable to that of SVMs for classification and regression. The main limitation of LS-SVM is that it lacks sparsity compared with SVMs, making LS-SVM unsuitable for handling large-scale data due to computation and memory costs. To obtain sparse LS-SVM, several pruning methods based on an iterative strategy were recently proposed but did not consider the quantity constraint on the number of reserved support vectors, as widely used in real-life applications. In this article, a noniterative algorithm is proposed based on the selection of globally representative points (global-representation-based sparse least squares support vector machine, GRS-LSSVM) to improve the performance of sparse LS-SVM. For the first time, we present a model of sparse LS-SVM with a quantity constraint. In solving the optimal solution of the model, we find that using globally representative points to construct the reserved support vector set produces a better solution than other methods. We design an indicator based on point density and point dispersion to evaluate the global representation of points in feature space. Using the indicator, the top globally representative points are selected in one step from all points to construct the reserved support vector set of sparse LS-SVM. After obtaining the set, the decision hyperplane of sparse LS-SVM is directly computed using an algebraic formula. This algorithm only consumes O(N2) in computational complexity and O(N) in memory cost which makes it suitable for large-scale data sets. The experimental results show that the proposed algorithm has higher sparsity, greater stability, and lower computational complexity than the traditional iterative algorithms.

Local Support Vector Machine Based on Cooperative Clustering for Very Large-Scale Dataset

MMSVC: an Efficient Unsupervised Learning Approach for Large-Scale Datasets.

A Multi-Class Large Margin Classifier

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

Large-scale support vector machine classification with redundant data reduction

One-class support vector machines for large-scale data sets

CLS-SVM: a local modeling method for time series forecasting

An adaptive support vector machine learning algorithm for large classification problem

Local Class Boundaries For Support Vector Machine

Twin Support Vector Machine with Local Structural Information for Pattern Classification

Large-Scale Subspace Clustering by Independent Distributed and Parallel Coding

A Learning and Classification Scheme of Large Training Set SVM Basd on NN-SRM Analysis

Classification Method Based on Global and Local Support Vector Machine

K-SVM: an Effective SVM Algorithm Based on K-means Clustering

Learnable Subspace Clustering

Stochastic gradient support vector machine with local structural information for pattern recognition

Structural Least Square Twin Support Vector Machine for Classification

A Confident Majority Voting Strategy for Parallel and Modular Support Vector Machines

Fast Support Vector Machine with Low-Computational Complexity for Large-Scale Classification

Noniterative Sparse LS-SVM Based on Globally Representative Point Selection

SVM Algorithms for Large Scale Classification Problems Based on Data Partition and Ensemble Learning