Abstract:Traditional Support Vector Machine (SVM) solution suffers from O(n2) time complexity, which makes it impractical to very large datasets. To reduce its high computational complexity, several data reduction methods are proposed in previous studies. However, such methods are not effective to extract informative patterns. In this paper, a two-stage informative pattern extraction approach is proposed. The first stage of our approach is data cleaning based on bootstrap sampling. A bundle of weak SVM classifiers are constructed on the sampled datasets. Training data correctly classified by all the weak classifiers are cleaned due to lacking useful information for training. To further extract more informative training data, two informative pattern extraction algorithms are proposed in the second stage. As most training data are eliminated and only the more informative samples remain, the final SVM training time is reduced significantly. Contributions of this paper are three-fold. (1) First, a parallelized bootstrap sampling based method is proposed to clean the initial training data. By doing that, a large number of training data with little information are eliminated. (2) Then, we present two algorithms to effectively extract more informative training data. Both algorithms are based on maximum information entropy according to the empirical misclassification probability of each sample estimated in the first stage. Therefore, training time can be further reduced for training data further reduction. (3) Finally, empirical studies on four large datasets show the effectiveness of our approach in reducing the training data size and the computational cost, compared with the state-of-the-art algorithms, including PEGASOS, LIBLINEAR SVM and RSVM. Meanwhile, the generalization performance of our approach is comparable with baseline methods.

A Method for Improving SVM Classifier by Excluding Redundant Information

Redundant Information in Support Vectors and Improved Support Vector Machine

One Kind of Reduced Learning Algorithm for Support Vector Domain Description

An Incremental Updating Method for Support Vector Machines

An updating method for support vector learning

Large-scale support vector machine classification with redundant data reduction

A New Method to Construct Reduced Vector Sets for Simplifying Support Vector Machines

An information-geometrical method for improving the performance of support vector machine classifiers

A hybrid method for speeding SVM training

Improved Feature Selection Algorithm Based on Svm and Correlation

A geometrical method to improve performance of the support vector machine.

Simplify decision function of reduced support vector machines

A dissimilarity based kernel space for support vector machines

A New Method of Sample Reduction for Support Vector Classification

Training Data Reduction to Speed Up SVM Training

Improving Support Vector Machine Classifiers By Modifying Kernel Functions

Simplified support vector machines via kernel-based clustering

Simplified Support Vector Machine Based on Reduced Vector Set Method

Pre-Extracting Method For Svm Classification Based On The Non-Parametric K-Nn Rule

Simplify Support Vector Machines by Iterative Learning

SVM-SVDD: a new method to solve data description problem with negative examples