Abstract:In recent years, short texts have become a kind of prevalent text on the internet. Due to the short length of each text, conventional topic models for short texts suffer from the sparsity of word co-occurrence information. Researchers have proposed different kinds of customized topic models for short texts by providing additional word co-occurrence information. However, these models cannot incorporate sufficient semantic word co-occurrence information and may bring additional noisy information. To address these issues, we propose a self-aggregated topic model incorporating document embeddings. Aggregating short texts into long documents according to document embeddings can provide sufficient word co-occurrence information and avoid incorporating non-semantic word co-occurrence information. However, document embeddings of short texts contain a lot of noisy information resulting from the sparsity of word co-occurrence information. So we discard noisy information by changing the document embeddings into global and local semantic information. The global semantic information is the similarity probability distribution on the entire dataset and the local semantic information is the distances of similar short texts. Then we adopt a nested Chinese restaurant process to incorporate these two kinds of information. Finally, we compare our model to several state-of-the-art models on four real-world short texts corpus. The experiment results show that our model achieves better performances in terms of topic coherence and classification accuracy.

Topic Discovery for Short Texts Using Word Embeddings

Short Text Understanding by Leveraging Knowledge into Topic Model.

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Topic Modeling over Short Texts by Incorporating Word Embeddings

A Correlated Topic Model Using Word Embeddings

Semantic Visualization for Short Texts with Word Embeddings

Topic Modeling Using Distributed Word Embeddings

Topic Discovery for Biomedical Corpus Using MeSH Embeddings

Exploiting Global Semantic Similarity Biterms for Short-Text Topic Discovery

Topic Discovery from Heterogeneous Texts

Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)

Research on Improve Topic Representation over Short Text.

Topic model based on co-occurrence word networks for unbalanced short text datasets

Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings

A CWTM Model of Topic Extraction for Short Text.

A neural topic model with word vectors and entity vectors for short texts

BTM: Topic Modeling over Short Texts

A Nested Chinese Restaurant Topic Model for Short Texts with Document Embeddings

Constructing Pseudo Documents With Semantic Similarity For Short Text Topic Discovery

TSSE-DMM: Topic Modeling for Short Texts Based on Topic Subdivision and Semantic Enhancement

Deriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection