Abstract:Ensemble methods use multiple classifiers to achieve better decisions than could be achieved using any of the constituent classifiers alone. However, both theoretical and experimental evidence have shown that very large ensembles are not necessarily superior, and small ensembles can often achieve better results. In this paper, we show how to combine a set of weak classifiers into a robust ensemble by using a joint sparse representation method, which assigns a sparse coefficient vector to the decision of each classifier. The sparse vector contains many zero entries, and thus the final ensemble only employs a small number of classifiers, corresponding to non-zero entries. Training data are partitioned into several sub-groups to generate sub-underdetermined systems. The joint sparse method enables these sub-groups to then share their information about individual classifiers, to obtain an improved overall classification. Partitioning the training dataset into subgroups makes the proposed joint sparse ensemble method parallelizable, making it suitable for large scale problems. In contrast, previous work on sparse approaches to ensemble learning was limited to datasets smaller than the number of classifiers. Two different strategies are described for generating the sub-underdetermined systems, and experiments show these to be effective when tested with two different data manipulation methods. Experiments evaluate the performance of the joint sparse ensemble learning method in comparison to five other state-of-the-art methods from the literature, each designed to train small and efficient ensembles. Results suggest that joint sparse ensemble learning outperforms other algorithms on most datasets.

Subspace Ensembles for Classification

A Novel Classifier Ensemble Method Based on Subspace Enhancement for High-Dimensional Data Classification.

Subspace Selective Ensemble Algorithm Based on Feature Clustering

A New Ensemble Method with Feature Space Partitioning for High-Dimensional Data Classification

Progressive Subspace Ensemble Learning

Feature weighted dual random sampling cluster Ensemble.

Feature subspace ensemble classifiers for microarray data

Semi-supervised Classifier Ensemble Model for High-Dimensional Data.

A Subspace Ensemble Framework for Classification with High Dimensional Missing Data

Adaptive Classifier Ensemble Method Based on Spatial Perception for High-Dimensional Data Classification

Support vector machine based ensemble classifier

A Novel Genetic Algorithm for Subspace Based Subclasssifier Selection

Joint Sparse Learning for Classification Ensemble.

Ensemble Method for Classification of High-Dimensional Data.

Adaptive Subspace Optimization Ensemble Method for High-Dimensional Imbalanced Data Classification

Ensembles of Feature Subspaces for Object Detection

A novel classifier ensemble method with sparsity and diversity

EROS: Ensemble Rough Subspaces

Adaptive Random Subspace Ensemble Classification Aided by X-means Clustering

Exploiting Ensemble Method in Semi-Supervised Learning

Classifier Ensemble Using A Heuristic Learning With Sparsity And Diversity