Abstract:Text clustering aims to organize a vast collection of documents into meaningful and coherent clusters, thereby facilitating the extraction of valuable insights. While current frameworks for text clustering try to minimize the anisotropy of pre-trained language models through contrastive learning of text embeddings, the approach of treating in-batch samples as negatives is suboptimal. The K-means algorithm offers a way to sample both hard negatives and false negatives. However, relying solely on a single measure of semantic similarity between distributions and using coarse-grained weighting for negative pairs may potentially limit performance. Furthermore, considering the very similar distribution in text clusters due to rich semantics, the Mahalanobis distance-based Gaussian Mixture Model (GMM) is prone to falling into local optima due to one Gaussian model, having a smaller weight, may gradually merging into another during the parameter evaluation by the EM algorithm. To tackle these challenges, we propose a model named JourTC: Jo int u nsupervised contrastive learning and r obust GMM for T ext C lustering. In the contrastive learning phase, hard negatives, potential false negatives, and their corresponding global similarity-aware weights are determined through posterior probabilities derived from a Robust GMM (RGMM). This RGMM utilizes the entropy of each individual Gaussian model as a metric and adaptively adjusts the posterior probabilities of samples based on the Gaussian models with both maximum and minimum entropy to diminish the influence of low-entropy Gaussian models. Extensive experiments have shown that JourTC can be seamlessly integrated into existing text clustering frameworks, leading to a notable improvement in accuracy. Our code is publicly available. 1

Cross Validation and Minimum Generation Error for Improved Model Clustering in HMM-based TTS

Cross-Validation and Minimum Generation Error Based Decision Tree Pruning for HMM-based Speech Synthesis

Full HMM Training for Minimizing Generation Error in Synthesis

Minimum Generation Error Training for HMM-Based Speech Synthesis

Modeling Pitch Trajectory by Hierarchical HMM with Minimum Generation Error Training.

HMM training method based on evolutionary computation and MDI in speech recognition

Model Adaptation for HMM-Based Speech Synthesis under Minimum Generation Error Criterion

Minimum Generation Error Training for HMM-based Prediction of Articulatory Movements

Improving the Performance of HMM-based Voice Conversion Using Context Clustering Decision Tree and Appropriate Regression Matrix Format.

A Full Training Framework of Cross-Stream Dependence Modelling for HMM-based Singing Voice Synthesis

Minimum Unit Selection Error Training for HMM-based Unit Selection Speech Synthesis System

Minimum Generation Error Training With Direct Log Spectral Distortion On Lsps For Hmm-Based Speech Synthesis

Text Prompted Speaker Verification Based On Phoneme Clustering With Earth Mover'S Distane And Cauchy-Schwarz Divergence

Perceptual Clustering Based Unit Selection Optimization for Concatenative Text-to-speech Synthesis

Joint unsupervised contrastive learning and robust GMM for text clustering

Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modeling

Minimum generation error training with weighted Euclidean distance on LSP for HMM-based speech synthesis

A Hierarchical Viterbi Algorithm For Mandarin Hybrid Speech Synthesis System

HAM-TTS: Hierarchical Acoustic Modeling for Token-Based Zero-Shot Text-to-Speech with Model and Data Scaling

Cross-stream Dependency Modeling Using Continuous F0 Model for HMM-based Speech Synthesis

HMM based speech synthesis with Global Variance Training method