Abstract:We study the topic of dimensionality reduction for $k$-means clustering. Dimensionality reduction encompasses the union of two approaches: \emph{feature selection} and \emph{feature extraction}. A feature selection based algorithm for $k$-means clustering selects a small subset of the input features and then applies $k$-means clustering on the selected features. A feature extraction based algorithm for $k$-means clustering constructs a small set of new artificial features and then applies $k$-means clustering on the constructed features. Despite the significance of $k$-means clustering as well as the wealth of heuristic methods addressing it, provably accurate feature selection methods for $k$-means clustering are not known. On the other hand, two provably accurate feature extraction methods for $k$-means clustering are known in the literature; one is based on random projections and the other is based on the singular value decomposition (SVD). This paper makes further progress towards a better understanding of dimensionality reduction for $k$-means clustering. Namely, we present the first provably accurate feature selection method for $k$-means clustering and, in addition, we present two feature extraction methods. The first feature extraction method is based on random projections and it improves upon the existing results in terms of time complexity and number of features needed to be extracted. The second feature extraction method is based on fast approximate SVD factorizations and it also improves upon the existing results in terms of time complexity. The proposed algorithms are randomized and provide constant-factor approximation guarantees with respect to the optimal $k$-means objective value.

Automatic Dimension Selection for a Non-negative Factorization Approach to Clustering Multiple Random Graphs

Penalized Nonnegative Nonnegative Matrix Tri-Factorization For Co-Clustering

Non-redundant Multiple Clustering by Nonnegative Matrix Factorization

Simultaneous Dimensionality and Complexity Model Selection for Spectral Graph Clustering

Automatic Parameter Selection for Non-Redundant Clustering

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

Rethinking Symmetric Matrix Factorization: A More General and Better Clustering Perspective

Simultaneous Bayesian Clustering and Model Selection with Mixture of Robust Factor Analyzers

Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection

Automatic Determination of Intrinsic Cluster Number Family in Spectral Clustering Using Random Walk on Graph.

Robust Graph Regularized Nonnegative Matrix Factorization for Clustering

Correntropy Based Graph Regularized Concept Factorization for Clustering

Automatic Non-negative Matrix Factorization Clustering with Competitive Sparseness Constraints

Discovering Multiple Co-Clusterings With Matrix Factorization

KNCFS: Feature selection for high-dimensional datasets based on improved random multi-subspace learning

Graph Regularized Sparse Non-Negative Matrix Factorization for Clustering

Multi-view clustering based on pairwise co-regularization and robust dual graph non-negative matrix factorization

Adaptive graph guided concept factorization on Grassmann manifold

Fine-grained Bipartite Concept Factorization for Clustering

Randomized Dimensionality Reduction for k-means Clustering

Clustering Time-Stamped Data Using Multiple Nonnegative Matrices Factorization