Abstract:Traditional Support Vector Machine (SVM) solution suffers from O(n2) time complexity, which makes it impractical to very large datasets. To reduce its high computational complexity, several data reduction methods are proposed in previous studies. However, such methods are not effective to extract informative patterns. In this paper, a two-stage informative pattern extraction approach is proposed. The first stage of our approach is data cleaning based on bootstrap sampling. A bundle of weak SVM classifiers are constructed on the sampled datasets. Training data correctly classified by all the weak classifiers are cleaned due to lacking useful information for training. To further extract more informative training data, two informative pattern extraction algorithms are proposed in the second stage. As most training data are eliminated and only the more informative samples remain, the final SVM training time is reduced significantly. Contributions of this paper are three-fold. (1) First, a parallelized bootstrap sampling based method is proposed to clean the initial training data. By doing that, a large number of training data with little information are eliminated. (2) Then, we present two algorithms to effectively extract more informative training data. Both algorithms are based on maximum information entropy according to the empirical misclassification probability of each sample estimated in the first stage. Therefore, training time can be further reduced for training data further reduction. (3) Finally, empirical studies on four large datasets show the effectiveness of our approach in reducing the training data size and the computational cost, compared with the state-of-the-art algorithms, including PEGASOS, LIBLINEAR SVM and RSVM. Meanwhile, the generalization performance of our approach is comparable with baseline methods.

A New Method of Sample Reduction for Support Vector Classification

Effective Large-scale Sample Reduction Strategy Based on Support Vector Machine

Sample Reduction Strategy for Support Vector Machines Based on Fisher Discriminant Analysis

One Kind of Reduced Learning Algorithm for Support Vector Domain Description

A Novel Parallel Reduced Support Vector Machine

A Fast Training Method for OC-SVM Based on the Random Sampling Lemma

Boundary Detection and Sample Reduction for One-Class Support Vector Machines

Novel Simplifying Method for Support Vector Machines and Its Application

Two Effective Sample Selection Methods for Support Vector Machine

Neighbors' distribution property and sample reduction for support vector machines

Large-scale support vector machine classification with redundant data reduction

Reduction Of Training Datasets Via Fuzzy Entropy For Support Vector Machines

New Method Based on Support Vector Machine in Classification for Hyperspectral Data

Training Data Reduction to Speed Up SVM Training

A New Method to Construct Reduced Vector Sets for Simplifying Support Vector Machines

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction

Simplified support vector machines via kernel-based clustering

A Fast Training Algorithm for Support Vector Machine Via Boundary Sample Selection

A New Sampling-Based Svm For Face Recognition

Sample Reduction for SVMs Via Data Structure Analysis.

A Kernel Clustering Algorithm for Fast Training of Support Vector Machines