Abstract:The identification of suitable feature subsets from High-Dimensional Low-Sample-Size (HDLSS) data is of paramount importance because this dataset often contains numerous redundant and irrelevant features, leading to poor classification performance. However, the selection of an optimal feature subset from a vast feature space creates a significant computational challenge. In the domain of HDLSS data, conventional feature selection methods often face challenges in achieving a balance between reducing the number of features and preserving high classification accuracy. Addressing these issues, the study introduces an effective framework that employs a filter and wrapper-based strategy specifically designed to address the classification challenges inherent in HDLSS data. The framework adopts a multi-step approach where ensemble feature selection integrates five filter ranking approaches: Chi-square ( ), Gini index (GI), F-score, Mutual Information (MI), and Symmetric uncertainty (SU) to identify the top-ranking features. In the subsequent stage, a wrapper-based search method is utilized, which employs the Differential Evaluation (DE) metaheuristic algorithm as the search strategy. The fitness of feature subsets during this search is assessed based on a weighted combination of the error rate of the Support Vector Machine (SVM) classifier and the ratio of feature cardinality. The datasets, after undergoing dimensionality reduction, are then utilized to construct classification models using SVM, K-Nearest Neighbors (KNN), and Logistic Regression (LR). The approach was evaluated on 13 HDLSS datasets to assess its efficacy in selecting appropriate feature subsets and improving Classification Accuracy (ACC) analog with Area Under the Curve (AUC). Results show that the proposed ensemble with wrapper-based approach produces a smaller number of features (ranging between 2 and 9 for all datasets), while maintaining a commendable average AUC and ACC (between 98% and 100%). The comparative analysis reveals that the proposed method surpasses both ensemble feature selection and non-feature selection approaches in terms of feature reduction and ACC. Additionally, when compared to various other state-of-the-art methods, this approach demonstrates commendable performance.

Feature selection inspired classifier ensemble reduction.

Integration of Classifier Diversity Measures for Feature Selection-Based Classifier Ensemble Reduction.

Feature Reduction Using Ensemble Approach.

A New Ensemble Feature Selection and Its Application to Pattern Classification

Subspace Selective Ensemble Algorithm Based on Feature Clustering

Classifier Subset Selection Based on Classifier Representation and Clustering Ensemble

An Ensemble Multi-Label Feature Selection Algorithm Based on Information Entropy

Feature Subset Selection for High-Dimensional, Low Sampling Size Data Classification Using Ensemble Feature Selection With a Wrapper-Based Search

Classifier Ensemble Selection Based on Affinity Propagation Clustering.

Feature selection for SVMs ensemble

An Ensemble of Classifiers with Genetic Algorithm Based Feature Selection

Selective Ensemble Based on Transformation of Classifiers Used Spca

Neural Network Ensemble Based on Rough Sets Reduction and Selective Strategy

Intuitionistic fuzzy kernel-matching pursuit ensemble algorithm based on hybrid selection strategy

Co-Regularized Ensemble for Feature Selection.

Experiments with Feature-Prior Hybrid Ensemble Method for Classification

An Ensemble Feature Selection Method for High-Dimensional Data Based on Sort Aggregation

A Novel Classifier Ensemble Method Based on Subspace Enhancement for High-Dimensional Data Classification.

Improvement on Classification Performance Based on Multiple Reduct Ensemble

A Novel Hierarchical Selective Ensemble Classifier with Bioinformatics Application

Ensemble Method for Classification of High-Dimensional Data.