Abstract:Unsupervised feature selection (UFS) is widely applied in machine learning and pattern recognition. However, most of the existing methods only consider a single sparsity, which makes it difficult to select valuable and discriminative feature subsets from the original high-dimensional feature set. In this paper, we propose a new UFS method called DSCOFS via embedding double sparsity constrained optimization into the classical principal component analysis (PCA) framework. Double sparsity refers to using $\ell_{2,0}$-norm and $\ell_0$-norm to simultaneously constrain variables, by adding the sparsity of different types, to achieve the purpose of improving the accuracy of identifying differential features. The core is that $\ell_{2,0}$-norm can remove irrelevant and redundant features, while $\ell_0$-norm can filter out irregular noisy features, thereby complementing $\ell_{2,0}$-norm to improve discrimination. An effective proximal alternating minimization method is proposed to solve the resulting nonconvex nonsmooth model. Theoretically, we rigorously prove that the sequence generated by our method globally converges to a stationary point. Numerical experiments on three synthetic datasets and eight real-world datasets demonstrate the effectiveness, stability, and convergence of the proposed method. In particular, the average clustering accuracy (ACC) and normalized mutual information (NMI) are improved by at least 3.34% and 3.02%, respectively, compared with the state-of-the-art methods. More importantly, two common statistical tests and a new feature similarity metric verify the advantages of double sparsity. All results suggest that our proposed DSCOFS provides a new perspective for feature selection.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: Most of the existing Unsupervised Feature Selection (UFS) methods only consider single sparsity, which makes it difficult to select valuable and discriminative feature subsets from the original high - dimensional feature set. Specifically: 1. **Single Sparsity Limitation**: Most existing methods use only one type of sparsity constraint (such as $\ell_2,0$-norm or $\ell_0$-norm), which makes it difficult for them to effectively remove irrelevant, redundant and noisy features when dealing with complex data. 2. **Poor Feature Selection Effect**: Due to the lack of multi - type sparsity constraints, the accuracy of existing methods in identifying differential features is insufficient, thus affecting the effect of feature selection. To solve these problems, the author proposes a new UFS method - DSCOFS (Double Sparsity Constrained Optimization for Feature Selection). This method enhances the effect of feature selection by embedding double - sparsity - constrained optimization in the classical Principal Component Analysis (PCA) framework. Double sparsity means using $\ell_2,0$-norm and $\ell_0$-norm to constrain variables simultaneously to improve the accuracy and discrimination of feature selection. Specifically: - **$\ell_2,0$-norm**: It is used to remove irrelevant and redundant features and ensure global structural sparsity. - **$\ell_0$-norm**: It is used to filter irregular noisy features and ensure local element sparsity. By combining these two sparsity constraints, DSCOFS can select valuable features more effectively and shows better clustering accuracy and Normalized Mutual Information (NMI) on multiple experimental data sets. The experimental results show that, compared with existing methods, DSCOFS improves the average Clustering Accuracy (ACC) and NMI by at least 3.34% and 3.02% respectively. In summary, this paper aims to improve the performance of unsupervised feature selection by introducing double - sparsity - constrained optimization, so as to better meet the feature selection challenges in high - dimensional data.

Enhancing Unsupervised Feature Selection via Double Sparsity Constrained Optimization

$$\Hbox {u}^2\hbox {f}^2\hbox {S}^2$$ U 2 F 2 S 2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection.

Feature Selection from High-Order Tensorial Data Via Sparse Decomposition

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

Simultaneous local clustering and unsupervised feature selection via strong space constraint

Unsupervised feature selection via dual space-based low redundancy scores and extended OLSDA

Double-Structured Sparsity Guided Flexible Embedding Learning for Unsupervised Feature Selection

Convex Sparse PCA for Unsupervised Feature Learning.

Unsupervised Feature Selection via Nonnegative Orthogonal Constrained Regularized Minimization

Sparse Tensor PCA via Tensor Decomposition for Unsupervised Feature Selection

Subspace Clustering Guided Unsupervised Feature Selection.

Unsupervised Feature Selection Algorithm Based on Sparse Representation

Rethinking Embedded Unsupervised Feature Selection: A Simple Joint Approach

Maximum Correntropy Criterion-Based Sparse Subspace Learning for Unsupervised Feature Selection

Unsupervised Feature Selection by Nonnegative Sparsity Adaptive Subspace Learning

Dependence Guided Unsupervised Feature Selection

Double-dictionary Learning Unsupervised Feature Selection Cooperating with Low-Rank and Sparsity

Half-Quadratic Minimization for Unsupervised Feature Selection on Incomplete Data.

Feature selection using symmetric uncertainty and hybrid optimization for high-dimensional data

Two-Dimensional Unsupervised Feature Selection via Sparse Feature Filter

Unsupervised feature selection for multi-cluster data