Abstract:Recent years have witnessed the unprecedented growth of online social media, resulting in short texts being the prevalent format of information on the Internet. Given the sparsity of data, however, short-text topic modeling remains a critical yet much-watched challenge in both academia and industry. Research has been devoted to building different types of probabilistic topic models for short texts, among which self-aggregation methods emerged recently to provide informative cross-text word co-occurrences. However, models along this line are still in their infancy and typically yield overfit results and exhibit high computational costs. In this paper, we propose a novel model called Pseudo-document-based Topic Model (PTM), which introduces the concept of pseudo-document to implicitly aggregate short texts against data sparsity. By modeling the topic distributions of latent pseudo-documents rather than short texts, PTM yields excellent performance in accuracy and efficiency. A word embedding-enhanced PTM (WE-PTM) is also proposed to leverage pre-trained word embeddings, which is essential to further alleviating data sparsity. Extensive experiments with self-aggregation or word embedding-based baselines on four real-world datasets including two online media short texts, demonstrate the high-quality topics learned by our models. Robustness to limited training samples and the explainable semantics of topics are also investigated.

Robust Word-Network Topic Model For Short Texts

Word Network Topic Model: A Simple but General Solution for Short and Imbalanced Texts

Short Text Understanding by Leveraging Knowledge into Topic Model.

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Topic model based on co-occurrence word networks for unbalanced short text datasets

Topic Modeling of Short Texts: A Pseudo-Document View with Word Embedding Enhancement.

Topic Model over Short Texts Incorporating Word Embedding

Short Text Topic Modeling With Flexible Word Patterns

Don't Forget the Quantifiable Relationship between Words: Using Recurrent Neural Network for Short Text Topic Discovery.

A CWTM Model of Topic Extraction for Short Text.

Topic Modeling over Short Texts by Incorporating Word Embeddings

Utilizing Recurrent Neural Network for Topic Discovery in Short Text Scenarios

A Self-adaptive Sliding Window Based Topic Model for Non-uniform Texts

Relational Biterm Topic Model: Short-Text Topic Modeling Using Word Embeddings

Context Reinforced Neural Topic Modeling over Short Texts

Learning Latent Topics from the Word Co-occurrence Network

Extracting Topics with SimultaneousWord Co-occurrence and Semantic Correlation Graphs: Neural Topic Modeling for Short Texts

Biterm Pseudo Document Topic Model for Short Text

BTM: Topic Modeling over Short Texts

Targeted Aspects Oriented Topic Modeling for Short Texts

Probabilistic Non-Negative Matrix Factorization and Its Robust Extensions for Topic Modeling.