Abstract:In this article, we investigate the use of a probabilistic model for unsupervised clustering in text collections. Unsupervised clustering has become a basic module for many intelligent text processing applications, such as information retrieval, text classification or information extraction.Recent proposals have been made of probabilistic clustering models, which build “soft” theme-document associations. These models allow to compute, for each document, a probability vector whose values can be interpreted as the strength of the association between documents and clusters. As such, these vectors can also serve to project texts into a lower-dimensional “semantic” space. These models however pose non-trivial estimation problems, which are aggravated by the very high dimensionality of the parameter space.The model considered in this paper consists of a mixture of multinomial distributions over the word counts, each component corresponding to a different theme. We propose a systematic evaluation framework to contrast various estimation procedures for this model. Starting with the expectation-maximization (EM) algorithm as the basic tool for inference, we discuss the importance of initialization and the influence of other features, such as the smoothing strategy or the size of the vocabulary, thereby illustrating the difficulties incurred by the high dimensionality of the parameter space. We empirically show that, in the case of text processing, these difficulties can be alleviated by introducing the vocabulary incrementally, due to the specific profile of the word count distributions. Using the fact that the model parameters can be analytically integrated out, we finally show that Gibbs sampling on the theme configurations is tractable and compares favorably to the basic EM approach.

Mixture Model-based Text Clustering：A Review

Document Clustering Based on Probabilistic Topic Model

A Mixture Language Model for the Classification of Chinese Online Reviews.

Inference and evaluation of the multinomial mixture model for text clustering

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

Clustering of Chinese Sentences Using the SMM Model

Document Clustering with Cluster Refinement and Model Selection Capabilities.

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Mixture Model Averaging for Clustering

Text Clustering Approach Based on Maximal Frequent Term Sets

Recent Developments in Model-Based Clustering with Applications

Evaluating mixture modeling for clustering: Recommendations and cautions.

Field independent probabilistic model for clustering multi-field documents

Towards Semantically Sensitive Text Clustering: a Feature Space Modeling Technology Based on Dimension Extension.

Challenges in model‐based clustering

Topic-based mixture language modelling

Short Text Clustering Based on Pitman-Yor Process Mixture Model

Grouped Text Clustering Using Non-Parametric Gaussian Mixture Experts

Exploring Performance of Clustering Methods on Document Sentiment Analysis

Conditional mixture modeling and model-based clustering

Model-based Clustering of Short Text Streams