Abstract:Feature selection prepares the AI-readiness of data by eliminating redundant features. Prior research falls into two primary categories: i) Supervised Feature Selection, which identifies the optimal feature subset based on their relevance to the target variable; ii) Unsupervised Feature Selection, which reduces the feature space dimensionality by capturing the essential information within the feature set instead of using target variable. However, SFS approaches suffer from time-consuming processes and limited generalizability due to the dependence on the target variable and downstream ML tasks. UFS methods are constrained by the deducted feature space is latent and untraceable. To address these challenges, we introduce an innovative framework for feature selection, which is guided by knockoff features and optimized through reinforcement learning, to identify the optimal and effective feature subset. In detail, our method involves generating "knockoff" features that replicate the distribution and characteristics of the original features but are independent of the target variable. Each feature is then assigned a pseudo label based on its correlation with all the knockoff features, serving as a novel metric for feature evaluation. Our approach utilizes these pseudo labels to guide the feature selection process in 3 novel ways, optimized by a single reinforced agent: 1). A deep Q-network, pre-trained with the original features and their corresponding pseudo labels, is employed to improve the efficacy of the exploration process in feature selection. 2). We introduce unsupervised rewards to evaluate the feature subset quality based on the pseudo labels and the feature space reconstruction loss to reduce dependencies on the target variable. 3). A new {\epsilon}-greedy strategy is used, incorporating insights from the pseudo labels to make the feature selection process more effective.

BoostFS: A Boosting-Based Irrelevant Feature Selection Algorithm

Feature Selection Based on Dependency Margin

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

Gradient Boosted Feature Selection.

Invariant optimal feature selection: A distance discriminant and feature ranking based solution

Bhattacharyya boosting

An Approximate Markov Blanket Feature Selection Algorithm

Boosting Bootstrap FLD Subspaces for Multiclass Problem

Robust twin boosting for feature selection from high-dimensional omics data with label noise

Feature selection based on artificial bee colony and gradient boosting decision tree

A novel hybrid feature selection method based on dynamic feature importance

MetaFS: An Effective Wrapper Feature Selection via Meta Learning

Feature Selection Based on Intrusive Outliers Rather Than All Instances

Feature Learning Viewpoint of AdaBoost and a New Algorithm

Optimizing Cuckoo Feature Selection Algorithm with the New Initialization Strategy and Fitness Function.

FS_SFS: A Novel Feature Selection Method for Support Vector Machines

Knockoff-Guided Feature Selection via A Single Pre-trained Reinforced Agent

Imbalanced Multifault Diagnosis via Improved Localized Feature Selection

Feature Selection for Imbalanced Data with Deep Sparse Autoencoders Ensemble

A Modified Sequential Deep Floating Search Algorithm For Feature Selection

AFS: An Attention-based mechanism for Supervised Feature Selection