Abstract:Supervised feature selection (FS) as an interpretable dimensionality reduction technique has received increasing attention, where linear discriminative analysis (LDA)-based method can select informative features discriminatively and obtain promising performance. When original data has more features than samples, however, LDA-based method generally encounters degradation since the appearance of irreversible scatter matrix. This situation is called the small sample size (SSS) problem. To overcome it and enhance the discriminant power of selected feature subsets, in this paper, we design an elegant LDA-based FS model referred to as Top-k Discriminative FS (TDFS), which is constructed by seamlessly integrating the ℓ2,0-norm equation constraint into uncorrelated LDA model. More concretely, the ℓ2,0-norm equation constraint can explicitly characterize the number of selective features k to ensure the sparsity of projected matrix and select top features. The uncorrelated LDA model aims to improve discriminative ability based on uncorrelated data in projected subspace. Given the formidable nature of solving this non-convex model, a novel optimization algorithm is further developed and the SSS problem can be efficaciously addressed during the optimization process. We first decompose projection matrix into a discrete selection matrix and its corresponding nonzero projection matrix, then concurrently optimize above two matrices by employing a column-by-column update scheme, during which the reversibility of scatter matrix in selective feature subspace can be easily guaranteed to solve SSS problem. The extensive experiments on four synthetic data sets and eight real-world data sets show that the proposed method outperforms eight competitors validated by three classifiers. Moreover, although the theoretical analysis proves that our algorithm has quartic time complexity on the number of selected features k, the running time experiments validate that TDFS is still efficient and applicable in scenarios where only a small number of features need to be selected. From above perspectives, our algorithm shows desirable performance to achieve discriminative FS.

Convergence analysis of sparse TSK fuzzy systems based on spectral Dai-Yuan conjugate gradient and application to high-dimensional feature selection

Embedded Feature Selection Approach Based on TSK Fuzzy System with Sparse Rule Base for High-Dimensional Classification Problems

Curse of Dimensionality for TSK Fuzzy Neural Networks: Explanation and Solutions

Top-k Discriminative Feature Selection with Uncorrelated and ℓ2,0-Norm Equation Constraints

A Concise TSK Fuzzy Ensemble Classifier Integrating Dropout and Bagging for High-dimensional Problems

A Faster Convergence and Concise Interpretability TSK Fuzzy Classifier Deep-Wide-based Integrated Learning

Automatic Spatial–Spectral Feature Selection for Hyperspectral Image Via Discriminative Sparse Multimodal Learning

Subspace Sparse Discriminative Feature Selection

Takagi-Sugeno-Kang Fuzzy System Fusion: A Survey at Hierarchical, Wide and Stacked Levels

A Split-Complex Valued Gradient-Based Descent Neuro-Fuzzy Algorithm for TS System and Its Convergence

Deep Clustering and Dynamic Routing Based TSK Fuzzy System for Classification

Smoothed L1/2 Regularizer Learning for Split-Complex Valued Neuro-Fuzzy Algorithm for TSK System and Its Convergence Results

Discriminative and Uncorrelated Feature Selection with Constrained Spectral Analysis in Unsupervised Learning

DG-ALETSK: A High-Dimensional Fuzzy Approach with Simultaneous Feature Selection and Rule Extraction

Bagging-AdaTSK: An Ensemble Fuzzy Classifier for High-Dimensional Data

Optimize TSK Fuzzy Systems for Regression Problems: Minibatch Gradient Descent with Regularization, DropRule, and AdaBound (MBGD-RDA)

Optimize TSK Fuzzy Systems for Regression Problems: Mini-Batch Gradient Descent with Regularization, DropRule and AdaBound (MBGD-RDA)

Iterative-Weighted Thresholding Method for Group-Sparsity-Constrained Optimization With Applications

Optimize TSK Fuzzy Systems for Big Data Classification Problems: Bag of Tricks.

Sparse Group Feature Selection by Weighted Thresholding Homotopy Method

Feature Selection Using Sparse Twin Bounded Support Vector Machine.