Abstract:Ensemble methods use multiple classifiers to achieve better decisions than could be achieved using any of the constituent classifiers alone. However, both theoretical and experimental evidence have shown that very large ensembles are not necessarily superior, and small ensembles can often achieve better results. In this paper, we show how to combine a set of weak classifiers into a robust ensemble by using a joint sparse representation method, which assigns a sparse coefficient vector to the decision of each classifier. The sparse vector contains many zero entries, and thus the final ensemble only employs a small number of classifiers, corresponding to non-zero entries. Training data are partitioned into several sub-groups to generate sub-underdetermined systems. The joint sparse method enables these sub-groups to then share their information about individual classifiers, to obtain an improved overall classification. Partitioning the training dataset into subgroups makes the proposed joint sparse ensemble method parallelizable, making it suitable for large scale problems. In contrast, previous work on sparse approaches to ensemble learning was limited to datasets smaller than the number of classifiers. Two different strategies are described for generating the sub-underdetermined systems, and experiments show these to be effective when tested with two different data manipulation methods. Experiments evaluate the performance of the joint sparse ensemble learning method in comparison to five other state-of-the-art methods from the literature, each designed to train small and efficient ensembles. Results suggest that joint sparse ensemble learning outperforms other algorithms on most datasets.

A Novel Dynamic Rough Subspace Based Selective Ensemble.

EROS: Ensemble Rough Subspaces

Rough set based cluster ensemble selection

Rough Reducts Based Svm Ensemble

Selective Ensemble under Regularization Framework

Ensemble selection with joint spectral clustering and structural sparsity

Novel Dynamic Ensemble Selection Algorithm Based on Local Optimal Ensembles

A Random Approximate Reduct-Based Ensemble Learning Approach and Its Application in Software Defect Prediction

Subspace Ensembles for Classification

Joint Sparse Learning for Classification Ensemble.

Using Deep Learning to Improve Ensemble Smoother: Applications to Subsurface Characterization

Soft Subspace Clustering Ensemble Framework Based on the Latent Model

Feature weighted dual random sampling cluster Ensemble.

A Novel Genetic Algorithm for Subspace Based Subclasssifier Selection

Feature selection inspired classifier ensemble reduction.

Support vector machine ensemble using rough sets theory

Subspace Learning for Unsupervised Feature Selection Via Adaptive Structure Learning and Rank Approximation

Simple Deep Random Model Ensemble

Resampling-based Selective Clustering Ensembles

LibD3C: Ensemble Classifiers with a Clustering and Dynamic Selection Strategy

Novel Selective Support Vector Machine Ensemble Learning Algorithm