Abstract:Ensemble methods use multiple classifiers to achieve better decisions than could be achieved using any of the constituent classifiers alone. However, both theoretical and experimental evidence have shown that very large ensembles are not necessarily superior, and small ensembles can often achieve better results. In this paper, we show how to combine a set of weak classifiers into a robust ensemble by using a joint sparse representation method, which assigns a sparse coefficient vector to the decision of each classifier. The sparse vector contains many zero entries, and thus the final ensemble only employs a small number of classifiers, corresponding to non-zero entries. Training data are partitioned into several sub-groups to generate sub-underdetermined systems. The joint sparse method enables these sub-groups to then share their information about individual classifiers, to obtain an improved overall classification. Partitioning the training dataset into subgroups makes the proposed joint sparse ensemble method parallelizable, making it suitable for large scale problems. In contrast, previous work on sparse approaches to ensemble learning was limited to datasets smaller than the number of classifiers. Two different strategies are described for generating the sub-underdetermined systems, and experiments show these to be effective when tested with two different data manipulation methods. Experiments evaluate the performance of the joint sparse ensemble learning method in comparison to five other state-of-the-art methods from the literature, each designed to train small and efficient ensembles. Results suggest that joint sparse ensemble learning outperforms other algorithms on most datasets.

Comparison among Methods of Ensemble Learning

Comparative analysis of ensemble learning methods for signal classification

A Comparison of Ensemble Methods in Financial Market Prediction.

An empirical comparison of ensemble classification algorithms with support vector machines

An experimental comparison of ensemble learning methods on decision boundaries

A Comparative Assessment of Ensemble Learning for Credit Scoring

Research on Ensemble Learning

Joint Sparse Learning for Classification Ensemble.

BoostTree and BoostForest for Ensemble Learning

Ensemble Methods: Foundations and Algorithms

Ensemble Transfer Learning Algorithm

A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

Ensemble learning: A survey

A novel ensemble classifier based on multiple diverse classification methods

A New Rotation Forest Ensemble Algorithm

Sentiment classification: The contribution of ensemble learning

Ensemble Learning Method for Classification: Integrating Data Envelopment Analysis with Machine Learning

Classifiers Combination with Observational Learning

MTForest: Ensemble Decision Trees based on Multi-Task Learning

Empirical analysis of support vector machine ensemble classifiers

Comparison of Ensemble Models as Solutions for Imbalanced Class Classification of Datasets