Abstract:The identification of suitable feature subsets from High-Dimensional Low-Sample-Size (HDLSS) data is of paramount importance because this dataset often contains numerous redundant and irrelevant features, leading to poor classification performance. However, the selection of an optimal feature subset from a vast feature space creates a significant computational challenge. In the domain of HDLSS data, conventional feature selection methods often face challenges in achieving a balance between reducing the number of features and preserving high classification accuracy. Addressing these issues, the study introduces an effective framework that employs a filter and wrapper-based strategy specifically designed to address the classification challenges inherent in HDLSS data. The framework adopts a multi-step approach where ensemble feature selection integrates five filter ranking approaches: Chi-square ( ), Gini index (GI), F-score, Mutual Information (MI), and Symmetric uncertainty (SU) to identify the top-ranking features. In the subsequent stage, a wrapper-based search method is utilized, which employs the Differential Evaluation (DE) metaheuristic algorithm as the search strategy. The fitness of feature subsets during this search is assessed based on a weighted combination of the error rate of the Support Vector Machine (SVM) classifier and the ratio of feature cardinality. The datasets, after undergoing dimensionality reduction, are then utilized to construct classification models using SVM, K-Nearest Neighbors (KNN), and Logistic Regression (LR). The approach was evaluated on 13 HDLSS datasets to assess its efficacy in selecting appropriate feature subsets and improving Classification Accuracy (ACC) analog with Area Under the Curve (AUC). Results show that the proposed ensemble with wrapper-based approach produces a smaller number of features (ranging between 2 and 9 for all datasets), while maintaining a commendable average AUC and ACC (between 98% and 100%). The comparative analysis reveals that the proposed method surpasses both ensemble feature selection and non-feature selection approaches in terms of feature reduction and ACC. Additionally, when compared to various other state-of-the-art methods, this approach demonstrates commendable performance.

An iterative SVM approach to feature selection and classification in high-dimensional datasets

Feature Selection and Parameter Optimization for Support Vector Machines: A New Approach Based on Genetic Algorithm with Feature Chromosomes.

Support Vector Machine-Recursive Feature Elimination for Localized Feature Selection

Gene Selection Using Gaussian Kernel Support Vector Machine Based Recursive Feature Elimination with Adaptive Kernel Width Strategy

A Joint Learning Framework for Optimal Feature Extraction and Multi-class SVM

A Hybrid Feature Selection Approach by Correlation-Based Filters and SVM-RFE

Parameters Selection in Gene Selection Using Gaussian Kernel Support Vector Machines by Genetic Algorithm

Accelerated Recursive Feature Elimination Based on Support Vector Machine for Key Variable Identification

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

Dimension Reduction of High-Dimensional Datasets Based on Stepwise SVM

Feature Selection Via Scaling Factor Integrated Multi-Class Support Vector Machines

SVM-Based Feature Selection for Differential Space Fusion and Its Application to Diabetic Fundus Image Classification

Discriminative Sparse Subspace Learning and Its Application to Unsupervised Feature Selection

A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors

Recursive support vector machines for dimensionality reduction.

A Correlation-Redundancy Guided Evolutionary Algorithm and Its Application to High-Dimensional Feature Selection in Classification

Multiclass feature selection algorithms base on R-SVM

A Novel Support Vector Classifier for Longitudinal High-dimensional Data and Its Application to Neuroimaging Data

Optimal Feature Selection in High-Dimensional Discriminant Analysis

Ultrahigh dimensional variable selection: beyond the linear model

Feature Subset Selection for High-Dimensional, Low Sampling Size Data Classification Using Ensemble Feature Selection With a Wrapper-Based Search