Abstract:Sparse regression based feature selection method has been extensively investigated these years. However, because it has a non-convex constraint, i.e., $\ell _{2,0}$ℓ2,0-norm constraint, this problem is very hard to solve. In this paper, unlike most of the other methods which only solve its slack version by introducing sparsity regularization into objective function forcibly, a novel framework is proposed by us to solve the original $\ell _{2,0}$ℓ2,0-norm constrained sparse regression based feature selection problem. We transform our objective function into Linear Discriminant Analysis (LDA) by using a new label coding method, thus enabling our model to calculate the ratio of inter-class scatter to intra-class scatter of features which is the most widely used feature discrimination evaluation metric. According to that ratio, features can be selected by a simple sorting method. The projection gradient descent algorithm is introduced to further improve the performance of our algorithm by using the solution obtained before as its initial solution. This ensures the stability of this iterative algorithm. We prove that the proposed method can get the global optimal solution of this non-convex problem when all features are statistically independent. For the general case where features are statistically dependent, extensive experiments on six small sample size datasets and one large-scale dataset show that our algorithm has comparable or better classification capability comparing with other eight state-of-the-art feature selection methods by the SVM classifier. We also show that our algorithm can obtain a low loss value, which means the solution of our algorithm can get very close to this NP-hard problem’s real solution. What is more, because we solve the original $\ell _{2,0}$ℓ2,0-norm constrained problem, we avoid the heavy work of tuning the regularization parameter because its meaning is explicit in our method, i.e., the number of selected features. At last, we evaluate the stability of our algorithm from two perspectives, i.e., the objective function values and the selected features, by experiments. From both perspectives, our algorithm shows satisfactory stability performance.

Two-timescale neurodynamic approaches to supervised feature selection based on alternative problem formulations

A neurodynamic optimization approach to supervised feature selection via fractional programming

Neurodynamics-driven supervised feature selection.

Supervised Feature Selection via Collaborative Neurodynamic Optimization

Adaptive Dual Graphs and Non-Convex Constraint Based Embedded Feature Selection

A Two-Timescale Duplex Neurodynamic Approach to Mixed-Integer Optimization

A New Alternating Suboptimal Dynamic Programming Algorithm with Applications for Feature Selection

Differential Evolution Based Feature Selection: A Niching-based Multi-objective Approach

Cardinality-constrained portfolio selection via two-timescale duplex neurodynamic optimization

TSFNFS: two-stage-fuzzy-neighborhood feature selection with binary whale optimization algorithm

Multi-objective integer programming approaches for solving optimal feature selection problem: a new perspective on multi-objective optimization problems in SBSE.

Joint nonlinear feature selection and continuous values regression network

Finding Optimal Diverse Feature Sets with Alternative Feature Selection

Incremental approaches for heterogeneous feature selection in dynamic ordered data

Feature Selection Using Double Parallel Feedforward Neural Networks And Particle Swarm Optimization

Multi-objective optimization algorithm based on clustering guided binary equilibrium optimizer and NSGA-III to solve high-dimensional feature selection problem

Unsupervised Feature Selection via Nonnegative Orthogonal Constrained Regularized Minimization

Safe Dynamic Sparse Training of Modified RBF Networks for Joint Feature Selection and Classification

Multiobjective Optimization Algorithm with Dynamic Operator Selection for Feature Selection in High-Dimensional Classification

Dynamic Incremental Optimization for Best Subset Selection