Abstract:Sparse support vector machine (SVM) is a popular classification technique that can simultaneously learn a small set of the most interpretable features and identify the support vectors. It has achieved great successes in many real-world applications. However, for large-scale problems involving a huge number of samples and ultra-high dimensional features, solving sparse SVMs remains challenging. By noting that sparse SVMs induce sparsities in both feature and sample spaces, we propose a novel approach, which is based on accurate estimations of the primal and dual optima of sparse SVMs, to simultaneously identify the inactive features and samples that are guaranteed to be irrelevant to the outputs. Thus, we can remove the identified inactive samples and features from the training phase, leading to substantial savings in the computational cost without sacrificing the accuracy. Moreover, we show that our method can be extended to multi-class sparse support vector machines. To the best of our knowledge, the proposed method is the \emph{first} \emph{static} feature and sample reduction method for sparse SVMs and multi-class sparse SVMs. Experiments on both synthetic and real data sets demonstrate that our approach significantly outperforms state-of-the-art methods and the speedup gained by our approach can be orders of magnitude.

Reduced Least Squares Support Vector Based On Kernel Partial Least Squares And Its Application Research

New Predictive Control Algorithms Based on Least Squares Support Vector Machines

Fuzzy Time Series Analysis Method Based on Least Squares Support Vector Machines

Sparse Representation Based on Projection Method in Online Least Squares Support Vector Machines

Least squares support vector machine based partially linear model identification

Dynamic least squares support vector machine

Using Svm To Model And Control Nonlinear Dynamical Systems

Modeling method of least squares support vector regression based on vector base learning

Least Squares Support Vector Machine with Self-Organizing Multiple Kernel Learning and Sparsity.

Sparse Least Squares Support Vector Machine for Function Estimation

One Kind of Reduced Learning Algorithm for Support Vector Domain Description

Dynamic Weighted Least Squares Support Vector Machines

Volumetric Error Prediction and Compensation of NC Machine Tool Based on Least Square Support Vector Machine

Robust Recursive Kernel Learning Modeling Method with Application to Blast Furnace

An efficient Kernel-based matrixized least squares support vector machine

Nonlinear Dynamic Process Monitoring Based on Kernel Partial Least Squares

Least Squares Support Vector Machine Algorithm

A Recursive Method of Identification of Hammerstein Model Based on Least Squares Support Vector Machines

Sparse Least Squares Low Rank Kernel Machines

A Non-Biased Form of Least Squares Support Vector Classifier and Its Fast Online Learning

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction