Abstract:Sparse support vector machine (SVM) is a popular classification technique that can simultaneously learn a small set of the most interpretable features and identify the support vectors. It has achieved great successes in many real-world applications. However, for large-scale problems involving a huge number of samples and ultra-high dimensional features, solving sparse SVMs remains challenging. By noting that sparse SVMs induce sparsities in both feature and sample spaces, we propose a novel approach, which is based on accurate estimations of the primal and dual optima of sparse SVMs, to simultaneously identify the inactive features and samples that are guaranteed to be irrelevant to the outputs. Thus, we can remove the identified inactive samples and features from the training phase, leading to substantial savings in the computational cost without sacrificing the accuracy. Moreover, we show that our method can be extended to multi-class sparse support vector machines. To the best of our knowledge, the proposed method is the \emph{first} \emph{static} feature and sample reduction method for sparse SVMs and multi-class sparse SVMs. Experiments on both synthetic and real data sets demonstrate that our approach significantly outperforms state-of-the-art methods and the speedup gained by our approach can be orders of magnitude.

Pre-extract support vectors based on projection center distance

Sparse Representation Based on Projection Method in Online Least Squares Support Vector Machines

Fast learning algorithm for support vector domain description

Fast extraction strategy of support vector machines

Image Feature Extraction Based on Support Vector Machine and Multi-DSP Combined Structure.

Fast incremental learning algorithm of linear support vector machine based on hull vectors

One Kind of Reduced Learning Algorithm for Support Vector Domain Description

Online support vector machines with vectors sieving method

A New Method of Sample Reduction for Support Vector Classification

A Transductive Learning Algorithm Based on Support Vector Machine

A Method for Vehicle Detection Based on Local Gradients Vector

Using Class-Center Vectors to Build Support Vector Machines

On-line Support Vector Machine Training Algorithm and Its Application

Neighbors' distribution property and sample reduction for support vector machines

Fast Pruning Superfluous Support Vectors in SVMs

Simplified Support Vector Machine Based on Reduced Vector Set Method

Improvements to Train Support Vector Machines Based on Convex Set Conception

A hybrid method for speeding SVM training

An Improved Support Vector Machine Using Class-Median Vectors

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction

Modeling method of least squares support vector regression based on vector base learning