Abstract:Feature selection plays a crucial role in classification tasks as part of the data preprocessing process. Effective feature selection can improve the robustness and interpretability of learning algorithms, and accelerate model learning. However, traditional statistical methods for feature selection are no longer practical in the context of high-dimensional data due to the computationally complex. Ensemble learning, a prominent learning method in machine learning, has demonstrated exceptional performance, particularly in classification problems. To address the issue, we propose a three-stage feature selection algorithm framework for high-dimensional data based on ensemble learning (EFS-GINI). Firstly, highly linearly correlated features are eliminated using the Spearman coefficient. Then, a feature selector based on the F-test is employed for the first stage selection. For the second stage, four feature subsets are formed using mutual information (MI), ReliefF, SURF, and SURF* filters in parallel. The third stage involves feature selection using a combinator based on GINI coefficient. Finally, a soft voting approach is proposed to employ for classification, including decision tree, naive Bayes, support vector machine (SVM), k-nearest neighbors (KNN) and random forest classifiers. To demonstrate the effectiveness and efficiency of the proposed algorithm, eight high-dimensional datasets are used and five feature selection methods are employed to compare with our proposed algorithm. Experimental results show that our method effectively enhances the accuracy and speed of feature selection. Moreover, to explore the biological significance of the proposed algorithm, we apply it on the renal cell carcinoma dataset GSE40435 from the Gene Expression Omnibus database. Two feature genes, NOP2 and NSUN5, are selected by our proposed algorithm. They are directly involved in regulating m5c RNA modification, which reveals the biological importance of EFS-GINI. Through bioinformatics analysis, we shows that m5C-related genes play an important role in the occurrence and progression of renal cell carcinoma, and are expected to become an important marker to predict the prognosis of patients.

Evaluation of Feature Selection Methods Using Bagging and Boosting Ensemble Techniques on High Throughput Biological Data

Support Vector Machines Ensemble With Optimizing Weights By Genetic Algorithm

Robust twin boosting for feature selection from high-dimensional omics data with label noise

FS–GBDT: identification multicancer-risk module via a feature selection algorithm by integrating Fisher score and GBDT

Comparison of Penalty-based Feature Selection Approach on High Throughput Biological Data.

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

An ensemble learning-based feature selection algorithm for identification of biomarkers of renal cell carcinoma

Feature selection based on artificial bee colony and gradient boosting decision tree

Improving the performance and interpretability on medical datasets using graphical ensemble feature selection

An Interactive Feature Selection Method Based on Learning-from-crowds

On Efficient Feature Ranking Methods for High-Throughput Data Analysis

Ensemble Feature Selection for Breast Cancer Classification using Microarray Data

Hybrid gene selection approach using XGBoost and multi-objective genetic algorithm for cancer classification

A practical tutorial on bagging and boosting based ensembles for machine learning: Algorithms, software tools, performance study, practical perspectives and opportunities

Two-level boosting classifiers ensemble based on feature selection for heart disease prediction

A Comparative Study of Ensemble Learning Approaches in the Classification of Breast Cancer Metastasis

A combinational feature selection and ensemble neural network method for classification of gene expression data

Exploring genomic feature selection: A comparative analysis of GWAS and machine learning algorithms in a large‐scale soybean dataset

A Hybrid Ensemble Algorithm Combining AdaBoost and Genetic Algorithm for Cancer Classification with Gene Expression Data.

An Adaptive Feature Selection Method for Microarray Data Analysis

Analysis of ensemble feature selection for correlated high-dimensional RNA-Seq cancer data