Abstract:An ensemble is a group of learners that work together as a committee to solve a problem. The existing ensemble learning algorithms often generate unnecessarily large ensembles, which consume extra computational resource and may degrade the generalization performance. Ensemble pruning algorithms aim to find a good subset of ensemble members to constitute a small ensemble, which saves the computational resource and performs as well as, or better than, the unpruned ensemble. This paper introduces a probabilistic ensemble pruning algorithm by choosing a set of ldquosparserdquo combination weights, most of which are zeros, to prune the ensemble. In order to obtain the set of sparse combination weights and satisfy the nonnegative constraint of the combination weights, a left-truncated, nonnegative, Gaussian prior is adopted over every combination weight. Expectation propagation (EP) algorithm is employed to approximate the posterior estimation of the weight vector. The leave-one-out (LOO) error can be obtained as a by-product in the training of EP without extra computation and is a good indication for the generalization error. Therefore, the LOO error is used together with the Bayesian evidence for model selection in this algorithm. An empirical study on several regression and classification benchmark data sets shows that our algorithm utilizes far less component learners but performs as well as, or better than, the unpruned ensemble. Our results are very competitive compared with other ensemble pruning algorithms.

Incremental learning based on ensemble pruning.

An ensemble learning method based on Bagging-SVM for intelligent sensor

Selective Bagging Based Incremental Learning

Hierarchical Bagging Ensemble Pruning Based on the Diversity of Base Learners

A Rough Sets Based Pruning Method for Bagging Ensemble

An Incremental Learning Algorithm Based on Support Vector Domain Classifier

Hierachical Bagging Ensemble Pruning Based on Deleting Worst Base Learner

Using Boosting to prune Double-Bagging ensembles

Incremental Classification Method Based on Ensemble

Margin Distribution Based Bagging Pruning

Ensemble Pruning Via Base-Classifier Replacement

Using Boosting and Clustering to Prune Bagging and Detect Noisy Data

Dynamic Weighting Ensembles for Incremental Learning

A Probabilistic Ensemble Pruning Algorithm

A Novel Bayesian Ensemble Pruning Method

Pareto Ensemble Pruning

Predictive Ensemble Pruning by Expectation Propagation

Margin-Based Pareto Ensemble Pruning: an Ensemble Pruning Algorithm That Learns to Search Optimized Ensembles.

Ensemble Pruning Via Individual Contribution Ordering

Instance-based Ensemble Pruning for Imbalanced Learning.

Increment Learning Algorithm Based on Bayesian Classifier Integration