Abstract:Supervised feature selection (FS) as an interpretable dimensionality reduction technique has received increasing attention, where linear discriminative analysis (LDA)-based method can select informative features discriminatively and obtain promising performance. When original data has more features than samples, however, LDA-based method generally encounters degradation since the appearance of irreversible scatter matrix. This situation is called the small sample size (SSS) problem. To overcome it and enhance the discriminant power of selected feature subsets, in this paper, we design an elegant LDA-based FS model referred to as Top-k Discriminative FS (TDFS), which is constructed by seamlessly integrating the ℓ2,0-norm equation constraint into uncorrelated LDA model. More concretely, the ℓ2,0-norm equation constraint can explicitly characterize the number of selective features k to ensure the sparsity of projected matrix and select top features. The uncorrelated LDA model aims to improve discriminative ability based on uncorrelated data in projected subspace. Given the formidable nature of solving this non-convex model, a novel optimization algorithm is further developed and the SSS problem can be efficaciously addressed during the optimization process. We first decompose projection matrix into a discrete selection matrix and its corresponding nonzero projection matrix, then concurrently optimize above two matrices by employing a column-by-column update scheme, during which the reversibility of scatter matrix in selective feature subspace can be easily guaranteed to solve SSS problem. The extensive experiments on four synthetic data sets and eight real-world data sets show that the proposed method outperforms eight competitors validated by three classifiers. Moreover, although the theoretical analysis proves that our algorithm has quartic time complexity on the number of selected features k, the running time experiments validate that TDFS is still efficient and applicable in scenarios where only a small number of features need to be selected. From above perspectives, our algorithm shows desirable performance to achieve discriminative FS.

EasyFS: an Efficient Model-free Feature Selection Framework via Elastic Transformation of Features

MetaFS: An Effective Wrapper Feature Selection via Meta Learning

Multi-label feature selection with high-sparse personalized and low-redundancy shared common features

Sparse Feature Selection Via Fast Embedding Spectral Analysis

An Efficient Framework for Unsupervised Feature Selection

Jointly evolving and compressing fuzzy system for feature reduction and classification

Unsupervised Feature Selection Based on Self-Representation Sparse Regression and Local Similarity Preserving

Principal Component Analysis With Fuzzy Elastic Net for Feature Selection

A New Adaptive Elastic Loss for Robust Unsupervised Feature Selection

Image Categorization Using ESFS: A New Embedded Feature Selection Method Based on SFS

ESFS: A new embedded feature selection method based on SFS

Compactness Score: a Fast Filter Method for Unsupervised Feature Selection

An Efficient Dual Feature Fusion Model of Feature Extraction for Hyperspectral Images

Py_FS: A Python Package for Feature Selection Using Meta-Heuristic Optimization Algorithms

Adaptive Dual Graphs and Non-Convex Constraint Based Embedded Feature Selection

Large-Scale Online Feature Selection for Ultra-High Dimensional Sparse Data.

FTDZOA: An Efficient and Robust FS Method with Multi-Strategy Assistance

Feature Selective Projection with Low-Rank Embedding and Dual Laplacian Regularization

High-dimensional Feature Selection for Power System Congestion Event Prognosis with Enhanced Evolutionary Computation

FS_SFS: A Novel Feature Selection Method for Support Vector Machines

Top-k Discriminative Feature Selection with Uncorrelated and ℓ2,0-Norm Equation Constraints