Abstract:Supervised feature selection (FS) as an interpretable dimensionality reduction technique has received increasing attention, where linear discriminative analysis (LDA)-based method can select informative features discriminatively and obtain promising performance. When original data has more features than samples, however, LDA-based method generally encounters degradation since the appearance of irreversible scatter matrix. This situation is called the small sample size (SSS) problem. To overcome it and enhance the discriminant power of selected feature subsets, in this paper, we design an elegant LDA-based FS model referred to as Top-k Discriminative FS (TDFS), which is constructed by seamlessly integrating the ℓ2,0-norm equation constraint into uncorrelated LDA model. More concretely, the ℓ2,0-norm equation constraint can explicitly characterize the number of selective features k to ensure the sparsity of projected matrix and select top features. The uncorrelated LDA model aims to improve discriminative ability based on uncorrelated data in projected subspace. Given the formidable nature of solving this non-convex model, a novel optimization algorithm is further developed and the SSS problem can be efficaciously addressed during the optimization process. We first decompose projection matrix into a discrete selection matrix and its corresponding nonzero projection matrix, then concurrently optimize above two matrices by employing a column-by-column update scheme, during which the reversibility of scatter matrix in selective feature subspace can be easily guaranteed to solve SSS problem. The extensive experiments on four synthetic data sets and eight real-world data sets show that the proposed method outperforms eight competitors validated by three classifiers. Moreover, although the theoretical analysis proves that our algorithm has quartic time complexity on the number of selected features k, the running time experiments validate that TDFS is still efficient and applicable in scenarios where only a small number of features need to be selected. From above perspectives, our algorithm shows desirable performance to achieve discriminative FS.

Unified dual-label semi-supervised learning with top-<i>k</i> feature selection

Unsupervised Feature Selection with Ordinal Locality.

Semi-Supervised Top-k Feature Selection with a General Optimization Framework

Semi-supervised feature selection based on structure and constraints preserving

Semi-supervised feature selection based on label propagation and subset selection

Research on Multi-Label Semi-Supervised Learning Algorithm Based on Dual Selection Criteria

Semi-supervised feature selection based on pairwise constraint-guided dual space latent representation learning and double sparse graphs discriminant

Unsupervised soft-label feature selection

A novel method of constrained feature selection by the measurement of pairwise constraints uncertainty

Sparse semi-supervised multi-label feature selection based on latent representation

Discriminative Semi-Supervised Feature Selection Via a Class-Credible Pseudo-Label Learning Framework

Semi-supervised feature selection based on discernibility matrix and mutual information

Soft Label Guided Unsupervised Discriminative Sparse Subspace Feature Selection

Effective Learning with Joint Discriminative and Representative Feature Selection

Top-k Discriminative Feature Selection with Uncorrelated and ℓ2,0-Norm Equation Constraints

Semi-supervised feature selection by minimum neighborhood redundancy and maximum neighborhood relevancy

Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning

Joint Semi-Supervised Feature Selection and Classification Through Bayesian Approach

Enhanced multi-label feature selection considering label-specific relevant information

Adaptive Collaborative Soft Label Learning for Unsupervised Multi-view Feature Selection

Adaptive Semi-Supervised Feature Selection for Cross-Modal Retrieval