Abstract:Text clustering aims to organize a vast collection of documents into meaningful and coherent clusters, thereby facilitating the extraction of valuable insights. While current frameworks for text clustering try to minimize the anisotropy of pre-trained language models through contrastive learning of text embeddings, the approach of treating in-batch samples as negatives is suboptimal. The K-means algorithm offers a way to sample both hard negatives and false negatives. However, relying solely on a single measure of semantic similarity between distributions and using coarse-grained weighting for negative pairs may potentially limit performance. Furthermore, considering the very similar distribution in text clusters due to rich semantics, the Mahalanobis distance-based Gaussian Mixture Model (GMM) is prone to falling into local optima due to one Gaussian model, having a smaller weight, may gradually merging into another during the parameter evaluation by the EM algorithm. To tackle these challenges, we propose a model named JourTC: Jo int u nsupervised contrastive learning and r obust GMM for T ext C lustering. In the contrastive learning phase, hard negatives, potential false negatives, and their corresponding global similarity-aware weights are determined through posterior probabilities derived from a Robust GMM (RGMM). This RGMM utilizes the entropy of each individual Gaussian model as a metric and adaptively adjusts the posterior probabilities of samples based on the Gaussian models with both maximum and minimum entropy to diminish the influence of low-entropy Gaussian models. Extensive experiments have shown that JourTC can be seamlessly integrated into existing text clustering frameworks, leading to a notable improvement in accuracy. Our code is publicly available. 1

Contrastive author-aware text clustering

Joint unsupervised contrastive learning and robust GMM for text clustering

SimCTC: A Simple Contrast Learning Method of Text Clustering (Student Abstract)

Contrastive Learning Subspace for Text Clustering

ECCT: Efficient Contrastive Clustering via Pseudo-Siamese Vision Transformer and Multi-view Augmentation

A lightweight mixup-based short texts clustering for contrastive learning

TeC: A Novel Method for Text Clustering with Large Language Models Guidance and Weakly-Supervised Contrastive Learning

Attentive Representation Learning with Adversarial Training for Short Text Clustering

Deep Temporal Contrastive Clustering

Text clustering on authorship attribution based on the features of punctuations usage

Mutual Clustering on Comparative Texts via Heterogeneous Information Networks

Self-supervised Document Clustering Based on BERT with Data Augment

AFSC: A self-supervised augmentation-free spatial clustering method based on contrastive learning for identifying spatial domains

Self-supervised Contrastive Attributed Graph Clustering

Contrastive Multiview Attribute Graph Clustering With Adaptive Encoders

Dual-Level Cross-Modal Contrastive Clustering

Graph Clustering with High-Order Contrastive Learning

Constrained Coclustering for Textual Documents.

Contrastive graph clustering via enhanced hard sample mining and cluster-guiding

CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering

An Evaluation on Feature Selection for Text Clustering