Abstract:We study the topic of dimensionality reduction for $k$-means clustering. Dimensionality reduction encompasses the union of two approaches: \emph{feature selection} and \emph{feature extraction}. A feature selection based algorithm for $k$-means clustering selects a small subset of the input features and then applies $k$-means clustering on the selected features. A feature extraction based algorithm for $k$-means clustering constructs a small set of new artificial features and then applies $k$-means clustering on the constructed features. Despite the significance of $k$-means clustering as well as the wealth of heuristic methods addressing it, provably accurate feature selection methods for $k$-means clustering are not known. On the other hand, two provably accurate feature extraction methods for $k$-means clustering are known in the literature; one is based on random projections and the other is based on the singular value decomposition (SVD). This paper makes further progress towards a better understanding of dimensionality reduction for $k$-means clustering. Namely, we present the first provably accurate feature selection method for $k$-means clustering and, in addition, we present two feature extraction methods. The first feature extraction method is based on random projections and it improves upon the existing results in terms of time complexity and number of features needed to be extracted. The second feature extraction method is based on fast approximate SVD factorizations and it also improves upon the existing results in terms of time complexity. The proposed algorithms are randomized and provide constant-factor approximation guarantees with respect to the optimal $k$-means objective value.

DSKmeans: A new kmeans-type approach to discriminative subspace clustering

Subspace Clustering by Directly Solving Discriminative K-means

Robust K-Subspace Discriminant Clustering.

Time Series K-Means: A New K-Means Type Smooth Subspace Clustering for Time Series Data.

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Discriminative K-Means Laplacian Clustering

Fuzzy K-Means Clustering With Discriminative Embedding

Robust Discriminant Subspace Clustering with Adaptive Local Structure Embedding.

An Entropy Weighting K-Means Algorithm for Subspace Clustering of High-Dimensional Sparse Data

Kernel k'-means algorithm for clustering analysis

Fast Adaptive K-Means Subspace Clustering for High-Dimensional Data

A modified k-means clustering with a density-sensitive distance metric

Randomized Dimensionality Reduction for k-means Clustering

DH-Kmeans: An Improved K-means Clustering Algorithm based on Dynamic Initial Cluster Center Determination and Hierarchical Clustering

Robust Discriminative Multi-View K-means Clustering with Feature Selection and Group Sparsity Learning

A Dimension Reduction Algorithm Preserving Both Global and Local Clustering Structure

A Fuzzy K-modes-based Algorithm for Soft Subspace Clustering

Multi-Scale Deep Subspace Clustering With Discriminative Learning

An optimized k-means algorithm of reducing cluster intra-dissimilarity for document clustering

Non-Redundant Subspace Clusterings with Nr-Kmeans and Nr-DipMeans

Extensions of Kmeans-Type Algorithms: A New Clustering Framework by Integrating Intracluster Compactness and Intercluster Separation