Abstract:The identification of suitable feature subsets from High-Dimensional Low-Sample-Size (HDLSS) data is of paramount importance because this dataset often contains numerous redundant and irrelevant features, leading to poor classification performance. However, the selection of an optimal feature subset from a vast feature space creates a significant computational challenge. In the domain of HDLSS data, conventional feature selection methods often face challenges in achieving a balance between reducing the number of features and preserving high classification accuracy. Addressing these issues, the study introduces an effective framework that employs a filter and wrapper-based strategy specifically designed to address the classification challenges inherent in HDLSS data. The framework adopts a multi-step approach where ensemble feature selection integrates five filter ranking approaches: Chi-square ( ), Gini index (GI), F-score, Mutual Information (MI), and Symmetric uncertainty (SU) to identify the top-ranking features. In the subsequent stage, a wrapper-based search method is utilized, which employs the Differential Evaluation (DE) metaheuristic algorithm as the search strategy. The fitness of feature subsets during this search is assessed based on a weighted combination of the error rate of the Support Vector Machine (SVM) classifier and the ratio of feature cardinality. The datasets, after undergoing dimensionality reduction, are then utilized to construct classification models using SVM, K-Nearest Neighbors (KNN), and Logistic Regression (LR). The approach was evaluated on 13 HDLSS datasets to assess its efficacy in selecting appropriate feature subsets and improving Classification Accuracy (ACC) analog with Area Under the Curve (AUC). Results show that the proposed ensemble with wrapper-based approach produces a smaller number of features (ranging between 2 and 9 for all datasets), while maintaining a commendable average AUC and ACC (between 98% and 100%). The comparative analysis reveals that the proposed method surpasses both ensemble feature selection and non-feature selection approaches in terms of feature reduction and ACC. Additionally, when compared to various other state-of-the-art methods, this approach demonstrates commendable performance.

WrapperRL: Reinforcement Learning Agent for Feature Selection in High-Dimensional Industrial Data

Industrial Fault Detection Based on Time-Frequency Distillation Autoencoder

Search Space Division Method for Wrapper Feature Selection on High-Dimensional Data Classification

ID-RDRL: a deep reinforcement learning-based feature selection intrusion detection model

Ensemble Wrapper Subsampling for Deep Modulation Classification

Network Intrusion Detection Through Discriminative Feature Selection by Using Sparse Logistic Regression

Deep Residual Principal Component Analysis As Feature Engineering for Industrial Data Analytics

DroidRL: Reinforcement Learning Driven Feature Selection for Android Malware Detection

Feature Subset Selection for High-Dimensional, Low Sampling Size Data Classification Using Ensemble Feature Selection With a Wrapper-Based Search

Efficient Wrapper Feature Selection using Autoencoder and Model Based Elimination

Exploiting the potential of deep reinforcement learning for classification tasks in high-dimensional and unstructured data

Explainable Deep Learning-Based Feature Selection and Intrusion Detection Method on the Internet of Things

IGRF-RFE: a hybrid feature selection method for MLP-based network intrusion detection on UNSW-NB15 dataset

Wrapper Feature Selection Algorithm for the Optimization of an Indicator System of Patent Value Assessment

Automated Feature Selection: A Reinforcement Learning Perspective

Model-driven deep unrolling: Towards interpretable deep learning against noise attacks for intelligent fault diagnosis

A Feature Learning Siamese Model for Intelligent Control of the Dynamic Range Compressor

Human-in-the-Loop Feature Selection Using Interpretable Kolmogorov-Arnold Network-based Double Deep Q-Network

Hybrid wrapper feature selection method based on genetic algorithm and extreme learning machine for intrusion detection

Deciphering the Feature Representation of Deep Neural Networks for High-Performance AI

Feature selection and feature learning for high-dimensional batch reinforcement learning: A survey