Abstract:The identification of suitable feature subsets from High-Dimensional Low-Sample-Size (HDLSS) data is of paramount importance because this dataset often contains numerous redundant and irrelevant features, leading to poor classification performance. However, the selection of an optimal feature subset from a vast feature space creates a significant computational challenge. In the domain of HDLSS data, conventional feature selection methods often face challenges in achieving a balance between reducing the number of features and preserving high classification accuracy. Addressing these issues, the study introduces an effective framework that employs a filter and wrapper-based strategy specifically designed to address the classification challenges inherent in HDLSS data. The framework adopts a multi-step approach where ensemble feature selection integrates five filter ranking approaches: Chi-square ( ), Gini index (GI), F-score, Mutual Information (MI), and Symmetric uncertainty (SU) to identify the top-ranking features. In the subsequent stage, a wrapper-based search method is utilized, which employs the Differential Evaluation (DE) metaheuristic algorithm as the search strategy. The fitness of feature subsets during this search is assessed based on a weighted combination of the error rate of the Support Vector Machine (SVM) classifier and the ratio of feature cardinality. The datasets, after undergoing dimensionality reduction, are then utilized to construct classification models using SVM, K-Nearest Neighbors (KNN), and Logistic Regression (LR). The approach was evaluated on 13 HDLSS datasets to assess its efficacy in selecting appropriate feature subsets and improving Classification Accuracy (ACC) analog with Area Under the Curve (AUC). Results show that the proposed ensemble with wrapper-based approach produces a smaller number of features (ranging between 2 and 9 for all datasets), while maintaining a commendable average AUC and ACC (between 98% and 100%). The comparative analysis reveals that the proposed method surpasses both ensemble feature selection and non-feature selection approaches in terms of feature reduction and ACC. Additionally, when compared to various other state-of-the-art methods, this approach demonstrates commendable performance.

Evaluation of Feature Selection Using Wrapper For Numeric Dataset With Random Forest Algorithm

A Novel Wrapper–Based Feature Selection for Early Diabetes Prediction Enhanced With a Metaheuristic

Empirical Evaluation of the Performance of Feature Selection Approaches on Random Forest

Critical Factor Analysis for prediction of Diabetes Mellitus using an Inclusive Feature Selection Strategy

Diabetes disease prediction using decision tree for feature selection

Evolution of the random subset feature selection algorithm for classification problem

Feature Selection using Random Forest Classifier for Predicting Prostate Cancer

Analyzing classification and feature selection strategies for diabetes prediction across diverse diabetes datasets

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

A Recognition Method for Diabetic Retinopathy Based on Feature Selection

Feature Selection Menggunakan Binary Wheal Optimizaton Algorithm (BWOA) pada Klasifikasi Penyakit Diabetes

Comparing different feature selection algorithms for cardiovascular disease prediction

A Novel Approach for Best Parameters Selection and Feature Engineering to Analyze and Detect Diabetes: Machine Learning Insights

Machine Learning Meets Meta-Heuristics: Bald Eagle Search Optimization and Red Deer Optimization for Feature Selection in Type II Diabetes Diagnosis

Feature Subset Selection for High-Dimensional, Low Sampling Size Data Classification Using Ensemble Feature Selection With a Wrapper-Based Search

Improving Performance of a Group of Classification Algorithms Using Resampling and Feature Selection

Feature Selection using Extra Trees for Breast Cancer Prediction

Heart failure prediction based on random forest algorithm using genetic algorithm for feature selectio

A Novel Feature Selection Method Based on MRMR and Enhanced Flower Pollination Algorithm for High Dimensional Biomedical Data

Nonparametric feature selection by random forests and deep neural networks

Hybrid feature selection and classification technique for early prediction and severity of diabetes type 2