Abstract:In this article, we investigate the use of a probabilistic model for unsupervised clustering in text collections. Unsupervised clustering has become a basic module for many intelligent text processing applications, such as information retrieval, text classification or information extraction.Recent proposals have been made of probabilistic clustering models, which build “soft” theme-document associations. These models allow to compute, for each document, a probability vector whose values can be interpreted as the strength of the association between documents and clusters. As such, these vectors can also serve to project texts into a lower-dimensional “semantic” space. These models however pose non-trivial estimation problems, which are aggravated by the very high dimensionality of the parameter space.The model considered in this paper consists of a mixture of multinomial distributions over the word counts, each component corresponding to a different theme. We propose a systematic evaluation framework to contrast various estimation procedures for this model. Starting with the expectation-maximization (EM) algorithm as the basic tool for inference, we discuss the importance of initialization and the influence of other features, such as the smoothing strategy or the size of the vocabulary, thereby illustrating the difficulties incurred by the high dimensionality of the parameter space. We empirically show that, in the case of text processing, these difficulties can be alleviated by introducing the vocabulary incrementally, due to the specific profile of the word count distributions. Using the fact that the model parameters can be analytically integrated out, we finally show that Gibbs sampling on the theme configurations is tractable and compares favorably to the basic EM approach.

Grouped Text Clustering Using Non-Parametric Gaussian Mixture Experts

Joint unsupervised contrastive learning and robust GMM for text clustering

A Multi-View Clustering Algorithm for Short Text

Text Clustering Using Ensemble Clustering Technique

A New Fuzzy Text Clustering Method Based On Nonnegative Factor Analysis

A Vector Reconstruction Based Clustering Algorithm Particularly for Large-Scale Text Collection

Improved GA-based Text Clustering Algorithm

Short text clustering based on word embeddings and EMD

Text-Guided Image Clustering

Word2Cluster: A New Multi-Label Text Clustering Algorithm with an Adaptive Clusters Number

A Model-Based Approach for Text Clustering with Outlier Detection

Text Clustering Approach Based on Maximal Frequent Term Sets

Text clustering based on term weights automatic partition

VRCA: A Clustering Algorithm for Massive Amount of Texts

A Novel Text Clustering Algorithm Based on Inner Product Space Model of Semantic

Text Clustering Via Particle Swarm Optimization

Inference and evaluation of the multinomial mixture model for text clustering

Text Clustering Using Vsm with Feature Clusters

Towards Semantically Sensitive Text Clustering: a Feature Space Modeling Technology Based on Dimension Extension.

A Linguistic Feature Based Text Clustering Method.

A New Text Clustering Method Using Hidden Markov Model