Abstract:Feature selection technique has become a basic but desired technique when analyzing high-throughput biological data due to its nature of large p and small n. In recent years, ensemble learning based feature selection methods have been widely proposed and studied. Ensemble methods employ multiple learning algorithms to obtain better predictive performance than any of the constituent learning algorithms separately. Also, the feature selected by ensemble classifiers can yield more accurate classification performance and more robust results. In our work, the bagging algorithm Random Forest (RF), and the boosting algorithms Gradient Boosting Decision Tree (GBDT), Extreme Gradient Boosting (XGBoost) are the main research objects. We compared the accuracy and robustness of three algorithms on six different datasets from TCGA database. Also, the three feature selection algorithms are further ensembled using a bagging procedure for the purpose of comparison with the original single classifier. The results of our work indicated that for single base feature selectors, boosting algorithms all outperform than bagging one in both performance and robustness. By applying the bagging-based feature selection procedure, the robustness of three single base feature selectors is improved significantly, but the accuracy of them is slightly reduced. GBDT with bagging-based feature selection procedure achieved the best performance using our proposed comprehensive metric which balances equally accuracy and robustness.

Investigating the Effect of Randomly Selected Feature Subsets on Bagging and Boosting

Support Vector Machines Ensemble With Optimizing Weights By Genetic Algorithm

A Weighted Subspace Approach for Improving Bagging Performance

Evaluation of Feature Selection Methods Using Bagging and Boosting Ensemble Techniques on High Throughput Biological Data

Bag of Little Bootstraps on Features for Enhancing Classification Performance.

Out-of-Bag Estimation of the Optimal Hyperparameter in SubBag Ensemble Method.

Subspace Ensembles for Classification

Using Boosting to prune Double-Bagging ensembles

Dbboost: Enhancing Imbalanced Classification by A Novel Ensemble Based Technique

A practical tutorial on bagging and boosting based ensembles for machine learning: Algorithms, software tools, performance study, practical perspectives and opportunities

Importance Feature Sampling in Random Subspace

A Rough Sets Based Pruning Method for Bagging Ensemble

Random Forests with Ensemble of Feature Spaces.

A Novel Genetic Algorithm for Subspace Based Subclasssifier Selection

Empirical analysis of support vector machine ensemble classifiers

An Empirical Study of Bagging Predictors for Imbalanced Data with Different Levels of Class Distribution

Bagging and Boosting Fine-tuning for Ensemble Learning

An empirical comparison of ensemble classification algorithms with support vector machines

BoostTree and BoostForest for Ensemble Learning

Multi-Classifier Ensemble Based on Random Feature Subspace

Experimental Study and Comparison of Imbalance Ensemble Classifiers with Dynamic Selection Strategy