Abstract:Text clustering aims to organize a vast collection of documents into meaningful and coherent clusters, thereby facilitating the extraction of valuable insights. While current frameworks for text clustering try to minimize the anisotropy of pre-trained language models through contrastive learning of text embeddings, the approach of treating in-batch samples as negatives is suboptimal. The K-means algorithm offers a way to sample both hard negatives and false negatives. However, relying solely on a single measure of semantic similarity between distributions and using coarse-grained weighting for negative pairs may potentially limit performance. Furthermore, considering the very similar distribution in text clusters due to rich semantics, the Mahalanobis distance-based Gaussian Mixture Model (GMM) is prone to falling into local optima due to one Gaussian model, having a smaller weight, may gradually merging into another during the parameter evaluation by the EM algorithm. To tackle these challenges, we propose a model named JourTC: Jo int u nsupervised contrastive learning and r obust GMM for T ext C lustering. In the contrastive learning phase, hard negatives, potential false negatives, and their corresponding global similarity-aware weights are determined through posterior probabilities derived from a Robust GMM (RGMM). This RGMM utilizes the entropy of each individual Gaussian model as a metric and adaptively adjusts the posterior probabilities of samples based on the Gaussian models with both maximum and minimum entropy to diminish the influence of low-entropy Gaussian models. Extensive experiments have shown that JourTC can be seamlessly integrated into existing text clustering frameworks, leading to a notable improvement in accuracy. Our code is publicly available. 1

A Text Clustering Algorithm Using an Online Clustering Scheme for Initialization

X-DMM: Fast and Scalable Model Based Text Clustering

A Model-Based Approach for Text Clustering with Outlier Detection

Text Clustering Approach Based on Maximal Frequent Term Sets

Joint unsupervised contrastive learning and robust GMM for text clustering

Grouped Text Clustering Using Non-Parametric Gaussian Mixture Experts

Improved GA-based Text Clustering Algorithm

A Vector Reconstruction Based Clustering Algorithm Particularly for Large-Scale Text Collection

Clustering Text Data Streams

Sampling Fuzzy K-Means Clustering Algorithm Based on Clonal Optimization

A Text Clustering Algorithm to Detect Basic Level Categories in Texts

An Adaptive Dirichlet Multinomial Mixture Model for Short Text Streaming Clustering

A hybrid approach using genetic algorithm and the differential evolution heuristic for enhanced initialization of the k-means algorithm with applications in text clustering

Incremental Algorithm of Text Soft Clustering

FGCH: a Fast and Grid Based Clustering Algorithm for Hybrid Data Stream

A Hash-Based Hierarchical Algorithm For Massive Text Clustering

A New Text Clustering Method Using Hidden Markov Model

Text clustering based on term weights automatic partition

A Context-Enhanced Dirichlet Model for Online Clustering in Short Text Streams.

An Incremental Algorithm of Text Clustering Based on Semantic Sequences

A New Agglomerative Hierarchical Clustering Algorithm Implementation Based on the Map Reduce Framework