Abstract:We study the topic of dimensionality reduction for $k$-means clustering. Dimensionality reduction encompasses the union of two approaches: \emph{feature selection} and \emph{feature extraction}. A feature selection based algorithm for $k$-means clustering selects a small subset of the input features and then applies $k$-means clustering on the selected features. A feature extraction based algorithm for $k$-means clustering constructs a small set of new artificial features and then applies $k$-means clustering on the constructed features. Despite the significance of $k$-means clustering as well as the wealth of heuristic methods addressing it, provably accurate feature selection methods for $k$-means clustering are not known. On the other hand, two provably accurate feature extraction methods for $k$-means clustering are known in the literature; one is based on random projections and the other is based on the singular value decomposition (SVD). This paper makes further progress towards a better understanding of dimensionality reduction for $k$-means clustering. Namely, we present the first provably accurate feature selection method for $k$-means clustering and, in addition, we present two feature extraction methods. The first feature extraction method is based on random projections and it improves upon the existing results in terms of time complexity and number of features needed to be extracted. The second feature extraction method is based on fast approximate SVD factorizations and it also improves upon the existing results in terms of time complexity. The proposed algorithms are randomized and provide constant-factor approximation guarantees with respect to the optimal $k$-means objective value.

Unsupervised Dimension Reduction Using Supervised Orthogonal Discriminant Projection for Clustering.

Unsupervised Feature Selection with Ordinal Locality.

Subspace Clustering by Directly Solving Discriminative K-means

Robust Discriminant Subspace Clustering with Adaptive Local Structure Embedding.

A General Framework for Dimensionality Reduction of K-Means Clustering

Local Feature Discriminant Projection

Randomized Dimensionality Reduction for k-means Clustering

Dimensionality-reduced subspace clustering

Unsupervised feature selection via dual space-based low redundancy scores and extended OLSDA

Unsupervised Linear Discriminant Analysis for Jointly Clustering and Subspace Learning

Adaptive Spectral Rotation via Joint Cluster and Pairwise Structure

Clustering Based on Supervised Learning of Exemplar Discriminative Information

Orthogonal Self-Guided Similarity Preserving Projection for Classification and Clustering.

Self-Supervised Graph Embedding Clustering

Semi-supervised dimensionality reduction using orthogonal projection divergence-based clustering for hyperspectral imagery

An Effective Clustering Optimization Method for Unsupervised Linear Discriminant Analysis

Learning-Augmented K-Means Clustering Using Dimensional Reduction

Discriminative K-Means Laplacian Clustering

Linear Semi-Supervised Dimensionality Reduction with Pairwise Constraint for Multiple Subclasses.

Unsupervised Neighborhood Component Analysis for Clustering.

Unsupervised Dimensionality Reduction Based on Fusing Multiple Clustering Results