Abstract:To efficiently deal with high-dimensional datasets in many areas, unsupervised feature selection (UFS) has become a rising technique for dimension reduction. Even though there are many UFS methods, most of them only consider the global structure of datasets by embedding a single sparse regularization or constraint. In this paper, we introduce a novel bi-sparse UFS method, called BSUFS, to simultaneously characterize both global and local structures. The core idea of BSUFS is to incorporate $\ell_{2,p}$-norm and $\ell_q$-norm into the classical principal component analysis (PCA), which enables our proposed method to select relevant features and filter out irrelevant noise accurately. Here, the parameters $p$ and $q$ are within the range of [0,1). Therefore, BSUFS not only constructs a unified framework for bi-sparse optimization, but also includes some existing works as special cases. To solve the resulting non-convex model, we propose an efficient proximal alternating minimization (PAM) algorithm using Riemannian manifold optimization and sparse optimization techniques. Theoretically, PAM is proven to have global convergence, i.e., for any random initial point, the generated sequence converges to a critical point that satisfies the first-order optimality condition. Extensive numerical experiments on synthetic and real-world datasets demonstrate the effectiveness of our proposed BSUFS. Specifically, the average accuracy (ACC) is improved by at least 4.71% and the normalized mutual information (NMI) is improved by at least 3.14% on average compared to the existing UFS competitors. The results validate the advantages of bi-sparse optimization in feature selection and show its potential for other fields in image processing. Our code will be available at <a class="link-external link-https" href="https://github.com/xianchaoxiu" rel="external noopener nofollow">this https URL</a>.

Unsupervised Feature Selection by Pareto Optimization

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

Supervised Feature Subset Selection with Ordinal Optimization

Unsupervised Feature Selection via Nonnegative Orthogonal Constrained Regularized Minimization

Unsupervised Feature Selection Via Maximum Projection and Minimum Redundancy

Convex Sparse PCA for Unsupervised Feature Learning.

Sparse PCA Via $L_{2,p}$-Norm Regularization for Unsupervised Feature Selection

Subset Selection by Pareto Optimization

Half-Quadratic Minimization for Unsupervised Feature Selection on Incomplete Data.

Feature subset selection via an improved discretization-based particle swarm optimization

Sparse and Flexible Projections for Unsupervised Feature Selection

Feature Selection of High Dimensional Data by Adaptive Potential Particle Swarm Optimization.

Unsupervised Feature Selection Algorithm Based on Sparse Representation

Sparse Representation Preserving for Unsupervised Feature Selection

Fast Sparse PCA via Positive Semidefinite Projection for Unsupervised Feature Selection

Unsupervised feature selection for multi-cluster data

Subset Selection by Pareto Optimization with Recombination

Bi-Sparse Unsupervised Feature Selection

Random Subspace Based Semi-Supervised Feature Selection.

Unsupervised Feature Analysis with Class Margin Optimization

Distribution Preserving Learning for Unsupervised Feature Selection