Abstract:The identification of suitable feature subsets from High-Dimensional Low-Sample-Size (HDLSS) data is of paramount importance because this dataset often contains numerous redundant and irrelevant features, leading to poor classification performance. However, the selection of an optimal feature subset from a vast feature space creates a significant computational challenge. In the domain of HDLSS data, conventional feature selection methods often face challenges in achieving a balance between reducing the number of features and preserving high classification accuracy. Addressing these issues, the study introduces an effective framework that employs a filter and wrapper-based strategy specifically designed to address the classification challenges inherent in HDLSS data. The framework adopts a multi-step approach where ensemble feature selection integrates five filter ranking approaches: Chi-square ( ), Gini index (GI), F-score, Mutual Information (MI), and Symmetric uncertainty (SU) to identify the top-ranking features. In the subsequent stage, a wrapper-based search method is utilized, which employs the Differential Evaluation (DE) metaheuristic algorithm as the search strategy. The fitness of feature subsets during this search is assessed based on a weighted combination of the error rate of the Support Vector Machine (SVM) classifier and the ratio of feature cardinality. The datasets, after undergoing dimensionality reduction, are then utilized to construct classification models using SVM, K-Nearest Neighbors (KNN), and Logistic Regression (LR). The approach was evaluated on 13 HDLSS datasets to assess its efficacy in selecting appropriate feature subsets and improving Classification Accuracy (ACC) analog with Area Under the Curve (AUC). Results show that the proposed ensemble with wrapper-based approach produces a smaller number of features (ranging between 2 and 9 for all datasets), while maintaining a commendable average AUC and ACC (between 98% and 100%). The comparative analysis reveals that the proposed method surpasses both ensemble feature selection and non-feature selection approaches in terms of feature reduction and ACC. Additionally, when compared to various other state-of-the-art methods, this approach demonstrates commendable performance.

Feature Selection Library (MATLAB Toolbox)

FS_SFS: A Novel Feature Selection Method for Support Vector Machines

FSinR: an exhaustive package for feature selection

FRI -- Feature Relevance Intervals for Interpretable and Interactive Data Exploration

Py_FS: A Python Package for Feature Selection Using Meta-Heuristic Optimization Algorithms

LOFS: Library of Online Streaming Feature Selection

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

TSFEL: Time Series Feature Extraction Library

Relief-Based Feature Selection: Introduction and Review

ESFS: A new embedded feature selection method based on SFS

Model-free feature selection to facilitate automatic discovery of divergent subgroups in tabular data

SLM: End-to-end Feature Selection via Sparse Learnable Masks

A comprehensive survey on feature selection in the various fields of machine learning

Variable Selection in Data Analysis: A Synthetic Data Toolkit

A novel hybrid feature selection method based on dynamic feature importance

A new computationally efficient algorithm to solve Feature Selection for Functional Data Classification in high-dimensional spaces

MetaFS: An Effective Wrapper Feature Selection via Meta Learning

Fibonacci and k-Subsecting Recursive Feature Elimination

Feature Subset Selection for High-Dimensional, Low Sampling Size Data Classification Using Ensemble Feature Selection With a Wrapper-Based Search

A Modified Sequential Deep Floating Search Algorithm For Feature Selection

Analysis and comparison of feature selection methods towards performance and stability