Abstract:Feature selection technique has become a basic but desired technique when analyzing high-throughput biological data due to its nature of large p and small n. In recent years, ensemble learning based feature selection methods have been widely proposed and studied. Ensemble methods employ multiple learning algorithms to obtain better predictive performance than any of the constituent learning algorithms separately. Also, the feature selected by ensemble classifiers can yield more accurate classification performance and more robust results. In our work, the bagging algorithm Random Forest (RF), and the boosting algorithms Gradient Boosting Decision Tree (GBDT), Extreme Gradient Boosting (XGBoost) are the main research objects. We compared the accuracy and robustness of three algorithms on six different datasets from TCGA database. Also, the three feature selection algorithms are further ensembled using a bagging procedure for the purpose of comparison with the original single classifier. The results of our work indicated that for single base feature selectors, boosting algorithms all outperform than bagging one in both performance and robustness. By applying the bagging-based feature selection procedure, the robustness of three single base feature selectors is improved significantly, but the accuracy of them is slightly reduced. GBDT with bagging-based feature selection procedure achieved the best performance using our proposed comprehensive metric which balances equally accuracy and robustness.

Using Boosting to prune Double-Bagging ensembles

DoubleEnsemble: A New Ensemble Method Based on Sample Reweighting and Feature Selection for Financial Data Analysis

BoostTree and BoostForest for Ensemble Learning

On the Size of Training Set and the Benefit from Ensemble

A practical tutorial on bagging and boosting based ensembles for machine learning: Algorithms, software tools, performance study, practical perspectives and opportunities

Saving Evaluation Time for the Decision Function in Boosting: Representation and Reordering Base Learner.

Evaluation of Feature Selection Methods Using Bagging and Boosting Ensemble Techniques on High Throughput Biological Data

A Meta Generalization Based Bagged Boosting Algorithm

Boosting Probabilistic Reasoning Model for Face Recognition

Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles

A Weighted Subspace Approach for Improving Bagging Performance

A Probabilistic Ensemble Pruning Algorithm

Boosting: An Ensemble Learning Tool for Compound Classification and QSAR Modeling

A Novel Bayesian Ensemble Pruning Method

Bagging-Based Selective Clusterer Ensemble

RotBoost: A technique for combining Rotation Forest and AdaBoost

To Bag is to Prune

Better Short than Greedy: Interpretable Models through Optimal Rule Boosting

Boosting Support Vector Machines Successfully.

Adaptive Ensemble Clustering With Boosting BLS-Based Autoencoder

Pareto Ensemble Pruning