Abstract:Traditional Support Vector Machine (SVM) solution suffers from O(n2) time complexity, which makes it impractical to very large datasets. To reduce its high computational complexity, several data reduction methods are proposed in previous studies. However, such methods are not effective to extract informative patterns. In this paper, a two-stage informative pattern extraction approach is proposed. The first stage of our approach is data cleaning based on bootstrap sampling. A bundle of weak SVM classifiers are constructed on the sampled datasets. Training data correctly classified by all the weak classifiers are cleaned due to lacking useful information for training. To further extract more informative training data, two informative pattern extraction algorithms are proposed in the second stage. As most training data are eliminated and only the more informative samples remain, the final SVM training time is reduced significantly. Contributions of this paper are three-fold. (1) First, a parallelized bootstrap sampling based method is proposed to clean the initial training data. By doing that, a large number of training data with little information are eliminated. (2) Then, we present two algorithms to effectively extract more informative training data. Both algorithms are based on maximum information entropy according to the empirical misclassification probability of each sample estimated in the first stage. Therefore, training time can be further reduced for training data further reduction. (3) Finally, empirical studies on four large datasets show the effectiveness of our approach in reducing the training data size and the computational cost, compared with the state-of-the-art algorithms, including PEGASOS, LIBLINEAR SVM and RSVM. Meanwhile, the generalization performance of our approach is comparable with baseline methods.

Sample Reduction Strategy for Support Vector Machines Based on Fisher Discriminant Analysis

Effective Large-scale Sample Reduction Strategy Based on Support Vector Machine

A New Method of Sample Reduction for Support Vector Classification

One Kind of Reduced Learning Algorithm for Support Vector Domain Description

Sample Reduction for SVMs Via Data Structure Analysis.

A Novel Parallel Reduced Support Vector Machine

A New Method to Construct Reduced Vector Sets for Simplifying Support Vector Machines

Novel Simplifying Method for Support Vector Machines and Its Application

Simplify decision function of reduced support vector machines

Training Data Reduction to Speed Up SVM Training

Information entropy based sample reduction for support vector data description.

Simplified Support Vector Machine Based on Reduced Vector Set Method

Boundary Detection and Sample Reduction for One-Class Support Vector Machines

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction

Fast extraction strategy of support vector machines

Large-scale support vector machine classification with redundant data reduction

A Cascade Method for Reducing Training Time and the Number of Support Vectors

A Discriminant Model for the Pattern Recognition of Linearly Independent Samples

Neighbors' distribution property and sample reduction for support vector machines

Discriminant analysis via support vectors

A hybrid method for speeding SVM training