Abstract:Sparse clustering, which aims to find a proper partition of an extremely high-dimensional data set with redundant noise features, has been attracted more and more interests in recent years. The existing studies commonly solve the problem in a framework of maximizing the weighted feature contributions subject to a $\ell_2/\ell_1$ penalty. Nevertheless, this framework has two serious drawbacks: One is that the solution of the framework unavoidably involves a considerable portion of redundant noise features in many situations, and the other is that the framework neither offers intuitive explanations on why this framework can select relevant features nor leads to any theoretical guarantee for feature selection consistency. In this article, we attempt to overcome those drawbacks through developing a new sparse clustering framework which uses a $\ell_{\infty}/\ell_0$ penalty. First, we introduce new concepts on optimal partitions and noise features for the high-dimensional data clustering problems, based on which the previously known framework can be intuitively explained in principle. Then, we apply the suggested $\ell_{\infty}/\ell_0$ framework to formulate a new sparse k-means model with the $\ell_{\infty}/\ell_0$ penalty ($\ell_0$-k-means for short). We propose an efficient iterative algorithm for solving the $\ell_0$-k-means. To deeply understand the behavior of $\ell_0$-k-means, we prove that the solution yielded by the $\ell_0$-k-means algorithm has feature selection consistency whenever the data matrix is generated from a high-dimensional Gaussian mixture model. Finally, we provide experiments with both synthetic data and the Allen Developing Mouse Brain Atlas data to support that the proposed $\ell_0$-k-means exhibits better noise feature detection capacity over the previously known sparse k-means with the $\ell_2/\ell_1$ penalty ($\ell_1$-k-means for short).

Penalized k-means algorithms for finding the correct number of clusters in a dataset

A penalized criterion for selecting the number of clusters for K-medians

Model-based Clustering with Nonconvex Penalty

Validity index for clustering with penalizing method

k is the Magic Number -- Inferring the Number of Clusters Through Nonparametric Concentration Inequalities

Hybrid k-Clustering: Blending k-Median and k-Center

Penalized Nonnegative Nonnegative Matrix Tri-Factorization For Co-Clustering

Penalized model-based clustering with cluster-specific diagonal covariance matrices and grouped variables

Outliers Detection Is Not So Hard: Approximation Algorithms for Robust Clustering Problems Using Local Search Techniques

Cluster-level Group Representativity Fairness in $k$-means Clustering

Relax and Merge: A Simple Yet Effective Framework for Solving Fair $k$-Means and $k$-sparse Wasserstein Barycenter Problems

What to Do When K-Means Clustering Fails: A Simple yet Principled Alternative Algorithm

Kernel k'-means algorithm for clustering analysis

Probably certifiably correct k-means clustering

Sparse K-Means with $\ell_{\infty}/\ell_0$ Penalty for High-Dimensional Data Clustering

Approximating Fair $k$-Min-Sum-Radii in Euclidean Space

A Graph-based Approach to Estimating the Number of Clusters

Parameterized Approximation for Robust Clustering in Discrete Geometric Spaces

Statistically Optimal K-means Clustering via Nonnegative Low-rank Semidefinite Programming

Parameterized Approximation Schemes for Clustering with General Norm Objectives

Self-adaptive Method of Determining Optimal Number of Clusters in Kernel-based Clustering Algorithm