Abstract:Text clustering aims to organize a vast collection of documents into meaningful and coherent clusters, thereby facilitating the extraction of valuable insights. While current frameworks for text clustering try to minimize the anisotropy of pre-trained language models through contrastive learning of text embeddings, the approach of treating in-batch samples as negatives is suboptimal. The K-means algorithm offers a way to sample both hard negatives and false negatives. However, relying solely on a single measure of semantic similarity between distributions and using coarse-grained weighting for negative pairs may potentially limit performance. Furthermore, considering the very similar distribution in text clusters due to rich semantics, the Mahalanobis distance-based Gaussian Mixture Model (GMM) is prone to falling into local optima due to one Gaussian model, having a smaller weight, may gradually merging into another during the parameter evaluation by the EM algorithm. To tackle these challenges, we propose a model named JourTC: Jo int u nsupervised contrastive learning and r obust GMM for T ext C lustering. In the contrastive learning phase, hard negatives, potential false negatives, and their corresponding global similarity-aware weights are determined through posterior probabilities derived from a Robust GMM (RGMM). This RGMM utilizes the entropy of each individual Gaussian model as a metric and adaptively adjusts the posterior probabilities of samples based on the Gaussian models with both maximum and minimum entropy to diminish the influence of low-entropy Gaussian models. Extensive experiments have shown that JourTC can be seamlessly integrated into existing text clustering frameworks, leading to a notable improvement in accuracy. Our code is publicly available. 1

Web Information Extraction Based on Clustering GHMM

Web Information Extraction Based on a Generalized Hidden Markov Model

Web object information extraction based on generalized hidden Markov model

Density-Based Clustering Algorithm for Hybrid Coding Detection in Search Engines

Dynamic Semantic Clustering Approach For Web User Interest

A New Text Clustering Method Using Hidden Markov Model

Enhanced Bayesian Gaussian hidden Markov mixture clustering for improved knowledge discovery

An online clustering algorithm for Chinese web snippets based on Generalized Suffix Array

Enhancing Web Text Clustering Accuracy and Efficiency With a Maximum Entropy Function Model: Overcoming High-Dimensional and Directional Challenges

A METHOD FOR WEB INFORMATION EXTRACTION BASED ON MULTI-LEARNING STRATEGIES

Clustering Heterogeneous Information Network by Joint Graph Embedding and Nonnegative Matrix Factorization

Joint unsupervised contrastive learning and robust GMM for text clustering

Dynamic Hierarchical Markov Random Fields for Integrated Web Data Extraction

Canonical Sequence Extraction And Hmm Model Building Based On Hierarchical Clustering

P2DHMM: A Novel Web Object Information Extraction Model

Dynamic Hierarchical Markov Random Fields and Their Application to Web Data Extraction

Exploring More Representative States of Hidden Markov Model in Optical Character Recognition: A Clustering-Based Model Pre-Training Approach

Combination of Static and Dynamic Modeling for Network Event Aggregation

Multiple Hypergraph Clustering of Web Images by MiningWord2Image Correlations.

Heterogeneous Transfer Learning for Image Clustering Via the SocialWeb.

Web Information Extraction Based on Probabilistic Model