Abstract:Ensemble methods use multiple classifiers to achieve better decisions than could be achieved using any of the constituent classifiers alone. However, both theoretical and experimental evidence have shown that very large ensembles are not necessarily superior, and small ensembles can often achieve better results. In this paper, we show how to combine a set of weak classifiers into a robust ensemble by using a joint sparse representation method, which assigns a sparse coefficient vector to the decision of each classifier. The sparse vector contains many zero entries, and thus the final ensemble only employs a small number of classifiers, corresponding to non-zero entries. Training data are partitioned into several sub-groups to generate sub-underdetermined systems. The joint sparse method enables these sub-groups to then share their information about individual classifiers, to obtain an improved overall classification. Partitioning the training dataset into subgroups makes the proposed joint sparse ensemble method parallelizable, making it suitable for large scale problems. In contrast, previous work on sparse approaches to ensemble learning was limited to datasets smaller than the number of classifiers. Two different strategies are described for generating the sub-underdetermined systems, and experiments show these to be effective when tested with two different data manipulation methods. Experiments evaluate the performance of the joint sparse ensemble learning method in comparison to five other state-of-the-art methods from the literature, each designed to train small and efficient ensembles. Results suggest that joint sparse ensemble learning outperforms other algorithms on most datasets.

Benchmarking Ensemble Classifiers with Novel Co-Trained Kernal Ridge Regression and Random Vector Functional Link Ensembles [Research Frontier].

A New Rotation Forest Ensemble Algorithm

A Variant of Rotation Forest for Constructing Ensemble Classifiers

An ensemble of decision trees with random vector functional link networks for multi-class classification.

Ensemble Classification Based On Random Linear Base Classifiers

A Novel Method for Constructing Ensemble Classifiers

Joint Sparse Learning for Classification Ensemble.

BoostTree and BoostForest for Ensemble Learning

An Attribute-Oriented Ensemble Classifier Based on Niche Gene Expression Programming

Diversified deep hierarchical kernel ensemble regression

Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [review Article]

An Optimal Random Projection k Nearest Neighbors Ensemble via Extended Neighborhood Rule for Binary Classification

Classifiers selection in ensembles using genetic algorithms for bankruptcy prediction

Leveraging Linear Independence of Component Classifiers: Optimizing Size and Prediction Accuracy for Online Ensembles

Ensemble Learning with Extremely Randomized k-Nearest Neighbors for Accurate and Efficient Classification

K-Dependence Bayesian Classifier Ensemble

Ensemble of optimal trees, random forest and random projection ensemble classification

BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning

Feature selection inspired classifier ensemble reduction.

A hybrid ensemble method with negative correlation learning for regression

Building Locally Discriminative Classifier Ensemble Through Classifier Fusion among Nearest Neighbors