Abstract:We study the clustering problem for mixtures of bounded covariance distributions, under a fine-grained separation assumption. Specifically, given samples from a $k$-component mixture distribution $D = \sum_{i =1}^k w_i P_i$, where each $w_i \ge \alpha$ for some known parameter $\alpha$, and each $P_i$ has unknown covariance $\Sigma_i \preceq \sigma^2_i \cdot I_d$ for some unknown $\sigma_i$, the goal is to cluster the samples assuming a pairwise mean separation in the order of $(\sigma_i+\sigma_j)/\sqrt{\alpha}$ between every pair of components $P_i$ and $P_j$. Our contributions are as follows: For the special case of nearly uniform mixtures, we give the first poly-time algorithm for this clustering task. Prior work either required separation scaling with the maximum cluster standard deviation (i.e. $\max_i \sigma_i$) [DKK+22b] or required both additional structural assumptions and mean separation scaling as a large degree polynomial in $1/\alpha$ [BKK22]. For general-weight mixtures, we point out that accurate clustering is information-theoretically impossible under our fine-grained mean separation assumptions. We introduce the notion of a clustering refinement -- a list of not-too-small subsets satisfying a similar separation, and which can be merged into a clustering approximating the ground truth -- and show that it is possible to efficiently compute an accurate clustering refinement of the samples. Furthermore, under a variant of the "no large sub-cluster'' condition from in prior work [BKK22], we show that our algorithm outputs an accurate clustering, not just a refinement, even for general-weight mixtures. As a corollary, we obtain efficient clustering algorithms for mixtures of well-conditioned high-dimensional log-concave distributions. Moreover, our algorithm is robust to $\Omega(\alpha)$-fraction of adversarial outliers.

Adaptative clustering by minimization of the mixing entropy criterion

High-dimensional logistic entropy clustering

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

Energy-Based Clustering: Fast and Robust Clustering of Data with Known Likelihood Functions

Entropy-Inspired Competitive Clustering Algorithms.

Clustering, classification, discriminant analysis, and dimension reduction via generalized hyperbolic mixtures

Clustering Based on Pairwise Distances When the Data is of Mixed Dimensions

Universal Lower Bounds and Optimal Rates: Achieving Minimax Clustering Error in Sub-Exponential Mixture Models

Clustering Mixtures of Bounded Covariance Distributions Under Optimal Separation

An Information-Theoretic Analysis of Hard and Soft Assignment Methods for Clustering

Noncommutative Model Selection for Data Clustering and Dimension Reduction Using Relative von Neumann Entropy

A unified framework for hard and soft clustering with regularized optimal transport

EM for Mixture of Linear Regression with Clustered Data

Beyond Labels: Advancing Cluster Analysis with the Entropy of Distance Distribution (EDD)

Mixtures of Gaussians and Minimum Relative Entropy Techniques for Modeling Continuous Uncertainties

Improving Entropy-Based Test-Time Adaptation from a Clustering View

Better than the best? Answers via model ensemble in density-based clustering

Optimal Clustering under Uncertainty

Flexible Clustering with a Sparse Mixture of Generalized Hyperbolic Distributions

Gamma-based clustering via ordered means with application to gene-expression analysis

Optimal Clustering of Discrete Mixtures: Binomial, Poisson, Block Models, and Multi-layer Networks