Abstract:Clustering is a fundamental tool in statistical machine learning in the presence of heterogeneous data. Most recent results focus primarily on optimal mislabeling guarantees when data are distributed around centroids with sub-Gaussian errors. Yet, the restrictive sub-Gaussian model is often invalid in practice since various real-world applications exhibit heavy tail distributions around the centroids or suffer from possible adversarial attacks that call for robust clustering with a robust data-driven initialization. In this paper, we present initialization and subsequent clustering methods that provably guarantee near-optimal mislabeling for general mixture models when the number of clusters and data dimensions are finite. We first introduce a hybrid clustering technique with a novel multivariate trimmed mean type centroid estimate to produce mislabeling guarantees under a weak initialization condition for general error distributions around the centroids. A matching lower bound is derived, up to factors depending on the number of clusters. In addition, our approach also produces similar mislabeling guarantees even in the presence of adversarial outliers. Our results reduce to the sub-Gaussian case in finite dimensions when errors follow sub-Gaussian distributions. To solve the problem thoroughly, we also present novel data-driven robust initialization techniques and show that, with probabilities approaching one, these initial centroid estimates are sufficiently good for the subsequent clustering algorithm to achieve the optimal mislabeling rates. Furthermore, we demonstrate that the Lloyd algorithm is suboptimal for more than two clusters even when errors are Gaussian and for two clusters when error distributions have heavy tails. Both simulated data and real data examples further support our robust initialization procedure and clustering algorithm.

Clustering Mixtures of Discrete Distributions: A Note on Mitra's Algorithm

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

Clustering Mixtures of Bounded Covariance Distributions Under Optimal Separation

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Robust Clustering Based on Finite Mixtures t Distribution

A Bayesian Approach to Clustering Matting Components in Spectral Matting

Clustering Mixtures with Almost Optimal Separation in Polynomial Time

Optimal Clustering of Discrete Mixtures: Binomial, Poisson, Block Models, and Multi-layer Networks

A rough set based clustering algorithm and the information theoretical approach to refine clusters

A provable initialization and robust clustering method for general mixture models

Spectral Clustering for Discrete Distributions

A Non-MCMC Procedure for Fitting Dirichlet Process Mixture Models

A Bayesian approach for clustering skewed data using mixtures of multivariate normal-inverse Gaussian distributions

Quantum Clustering and Gaussian Mixtures

Mixture Model-based Text Clustering：A Review

Analysis of spectral clustering algorithms for community detection: the general bipartite setting

Revisiting k-means: New Algorithms via Bayesian Nonparametrics

Research on mixture language model-based document clustering

Mixture Model Averaging for Clustering

Semidefinite programming on population clustering: a global analysis

Identifying the number of clusters in discrete mixture models