Abstract:We study the topic of dimensionality reduction for $k$-means clustering. Dimensionality reduction encompasses the union of two approaches: \emph{feature selection} and \emph{feature extraction}. A feature selection based algorithm for $k$-means clustering selects a small subset of the input features and then applies $k$-means clustering on the selected features. A feature extraction based algorithm for $k$-means clustering constructs a small set of new artificial features and then applies $k$-means clustering on the constructed features. Despite the significance of $k$-means clustering as well as the wealth of heuristic methods addressing it, provably accurate feature selection methods for $k$-means clustering are not known. On the other hand, two provably accurate feature extraction methods for $k$-means clustering are known in the literature; one is based on random projections and the other is based on the singular value decomposition (SVD). This paper makes further progress towards a better understanding of dimensionality reduction for $k$-means clustering. Namely, we present the first provably accurate feature selection method for $k$-means clustering and, in addition, we present two feature extraction methods. The first feature extraction method is based on random projections and it improves upon the existing results in terms of time complexity and number of features needed to be extracted. The second feature extraction method is based on fast approximate SVD factorizations and it also improves upon the existing results in terms of time complexity. The proposed algorithms are randomized and provide constant-factor approximation guarantees with respect to the optimal $k$-means objective value.

Mixed-type Distance Shrinkage and Selection for Clustering via Kernel Metric Learning

Mixed-Type Distance Shrinkage and Selection for Clustering via Kernel Metric Learning

Distance-Based Classifier Via the Kernel Trick.

Benchmarking distance-based partitioning methods for mixed-type data

A k-mean clustering algorithm for mixed numeric and categorical data

The Exploitation of Distance Distributions for Clustering

Learning Metrics via Discriminant Kernels and Multidimensional Scaling: Toward Expected Euclidean Representation

KPML: A Novel Probabilistic Perspective Kernel Mahalanobis Distance Metric Learning Model for Semi-supervised Clustering

Learning Distance Metric for Support Vector Machine: A Multiple Kernel Learning Approach.

An Investigation into Distance Measures in Cluster Analysis

Kernel Density Metric Learning

Subspace Clustering by Directly Solving Discriminative K-means

Efficient Clustering with Limited Distance Information

Sub-One Quasi-Norm-Based k-Means Clustering Algorithm and Analyses

K-Centers Algorithm for Clustering Mixed Type Data

A Kernel Classification Framework for Metric Learning.

Data Clustering: Integrating Different Distance Measures with Modified k-Means Algorithm

Nonlinear Metric Learning with Kernel Density Estimation

Clustering Based on Pairwise Distances When the Data is of Mixed Dimensions

Randomized Dimensionality Reduction for k-means Clustering

ThetA -- fast and robust clustering via a distance parameter