Abstract:The identification of suitable feature subsets from High-Dimensional Low-Sample-Size (HDLSS) data is of paramount importance because this dataset often contains numerous redundant and irrelevant features, leading to poor classification performance. However, the selection of an optimal feature subset from a vast feature space creates a significant computational challenge. In the domain of HDLSS data, conventional feature selection methods often face challenges in achieving a balance between reducing the number of features and preserving high classification accuracy. Addressing these issues, the study introduces an effective framework that employs a filter and wrapper-based strategy specifically designed to address the classification challenges inherent in HDLSS data. The framework adopts a multi-step approach where ensemble feature selection integrates five filter ranking approaches: Chi-square ( ), Gini index (GI), F-score, Mutual Information (MI), and Symmetric uncertainty (SU) to identify the top-ranking features. In the subsequent stage, a wrapper-based search method is utilized, which employs the Differential Evaluation (DE) metaheuristic algorithm as the search strategy. The fitness of feature subsets during this search is assessed based on a weighted combination of the error rate of the Support Vector Machine (SVM) classifier and the ratio of feature cardinality. The datasets, after undergoing dimensionality reduction, are then utilized to construct classification models using SVM, K-Nearest Neighbors (KNN), and Logistic Regression (LR). The approach was evaluated on 13 HDLSS datasets to assess its efficacy in selecting appropriate feature subsets and improving Classification Accuracy (ACC) analog with Area Under the Curve (AUC). Results show that the proposed ensemble with wrapper-based approach produces a smaller number of features (ranging between 2 and 9 for all datasets), while maintaining a commendable average AUC and ACC (between 98% and 100%). The comparative analysis reveals that the proposed method surpasses both ensemble feature selection and non-feature selection approaches in terms of feature reduction and ACC. Additionally, when compared to various other state-of-the-art methods, this approach demonstrates commendable performance.

A New Feature Selection Method Based on a Validity Index of Feature Subset

A New Validity Index of Feature Subset for Evaluating the Dimensionality Reduction Algorithms

$$\Hbox {u}^2\hbox {f}^2\hbox {S}^2$$ U 2 F 2 S 2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection.

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

Feature Selection Method on Imbalanced Text

A feature selection algorithm based on classification complementarity

A Two-Phase Feature Selection Method Using Both Filter and Wrapper

An Uncertainty-Accuracy-based Score Function for Wrapper Methods in Feature Selection

Wrapper Feature Selection Algorithm for the Optimization of an Indicator System of Patent Value Assessment

An Approximate Markov Blanket Feature Selection Algorithm

Feature Subset Selection for High-Dimensional, Low Sampling Size Data Classification Using Ensemble Feature Selection With a Wrapper-Based Search

FS_SFS: A Novel Feature Selection Method for Support Vector Machines

Invariant optimal feature selection: A distance discriminant and feature ranking based solution

Search Space Division Method for Wrapper Feature Selection on High-Dimensional Data Classification

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

Heuristic feature selection method for clustering

Feature Selection Approach Based on Improved Fuzzy C-Means with Principle of Refined Justifiable Granularity

A novel hybrid feature selection method based on dynamic feature importance

An Adaptive Feature Selection Method for Multi-Class Classification.

ESFS: A new embedded feature selection method based on SFS

MetaFS: An Effective Wrapper Feature Selection via Meta Learning