Abstract:Sparse regression based feature selection method has been extensively investigated these years. However, because it has a non-convex constraint, i.e., $\ell _{2,0}$ℓ2,0-norm constraint, this problem is very hard to solve. In this paper, unlike most of the other methods which only solve its slack version by introducing sparsity regularization into objective function forcibly, a novel framework is proposed by us to solve the original $\ell _{2,0}$ℓ2,0-norm constrained sparse regression based feature selection problem. We transform our objective function into Linear Discriminant Analysis (LDA) by using a new label coding method, thus enabling our model to calculate the ratio of inter-class scatter to intra-class scatter of features which is the most widely used feature discrimination evaluation metric. According to that ratio, features can be selected by a simple sorting method. The projection gradient descent algorithm is introduced to further improve the performance of our algorithm by using the solution obtained before as its initial solution. This ensures the stability of this iterative algorithm. We prove that the proposed method can get the global optimal solution of this non-convex problem when all features are statistically independent. For the general case where features are statistically dependent, extensive experiments on six small sample size datasets and one large-scale dataset show that our algorithm has comparable or better classification capability comparing with other eight state-of-the-art feature selection methods by the SVM classifier. We also show that our algorithm can obtain a low loss value, which means the solution of our algorithm can get very close to this NP-hard problem’s real solution. What is more, because we solve the original $\ell _{2,0}$ℓ2,0-norm constrained problem, we avoid the heavy work of tuning the regularization parameter because its meaning is explicit in our method, i.e., the number of selected features. At last, we evaluate the stability of our algorithm from two perspectives, i.e., the objective function values and the selected features, by experiments. From both perspectives, our algorithm shows satisfactory stability performance.

Sparse Group Feature Selection by Weighted Thresholding Homotopy Method

Ventilation-perfusion relationships in children with heart disease and diminished pulmonary blood flow.

Feature Selection from High-Order Tensorial Data Via Sparse Decomposition

Structured Sparse Model Based Feature Selection and Classification for Hyperspectral Imagery

Exclusive Sparsity Norm Minimization with Random Groups Via Cone Projection

Subspace Newton method for sparse group optimization problem

A minimax optimal approach to high-dimensional double sparse linear regression

Optimal Feature Selection for Sparse Linear Discriminant Analysis and Its Applications in Gene Expression Data.

Constrained Mix Sparse Optimization via Hard Thresholding Pursuit

Weighted Group Sparsity-Constrained Tensor Factorization for Hyperspectral Unmixing

Local Similarity Constraint-Based Sparse Algorithm for Hyperspectral Target Detection

Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization.

Convergence analysis of sparse TSK fuzzy systems based on spectral Dai-Yuan conjugate gradient and application to high-dimensional feature selection

Group sparse sufficient dimension reduction: a model-free group variable selection method

Half-Space Proximal Stochastic Gradient Method for Group-Sparsity Regularized Problem

Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization

Online Weighted Multi-task Feature Selection

Iterative Hard Thresholding for Weighted Sparse Approximation

Homogeneity and Sparsity Analysis for High-Dimensional Panel Data Models

Iterative Soft / Hard Thresholding Homotopy Algorithm for Sparse Recovery