Abstract:Topic models have shown great promise in discovering latent semantic structures from complex data corpora, ranging from text documents and web news articles to images, videos, and even biological data. In order to deal with massive data collections and dynamic text streams, probabilistic online topic models such as online latent Dirichlet allocation (OLDA) have recently been developed. However, due to normalization constraints, OLDA can be ineffective in controlling the sparsity of discovered representations, a desirable property for learning interpretable semantic patterns, especially when the total number of topics is large. In contrast, sparse topical coding (STC) has been successfully introduced as a non-probabilistic topic model for effectively discovering sparse latent patterns by using sparsity-inducing regularization. But, unfortunately STC cannot scale to very large datasets or deal with online text streams, partly due to its batch learning procedure. In this paper, we present a sparse online topic model, which directly controls the sparsity of latent semantic patterns by imposing sparsity-inducing regularization and learns the topical dictionary by an online algorithm. The online algorithm is efficient and guaranteed to converge. Extensive empirical results of the sparse online topic model as well as its collapsed and supervised extensions on a large-scale Wikipedia dataset and the medium-sized 20Newsgroups dataset demonstrate appealing performance.

Efficient Topic Modeling on Phrases via Sparsity

Short Text Understanding by Leveraging Knowledge into Topic Model.

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

Parsimonious Topic Models with Salient Word Discovery

Short Text Topic Modeling With Flexible Word Patterns

Sparse online topic models

Sparse Relational Topic Models for Document Networks.

Efficient Methods for Incorporating Knowledge into Topic Models

Sys-TM: A Fast and General Topic Modeling System

Topic Modeling over Short Texts by Incorporating Word Embeddings

A Hierarchical Latent Topic Model Based on Sparse Coding

PhraseCTM: Correlated Topic Modeling on Phrases Within Markov Random Fields.

Let the Pretrained Language Models "Imagine" for Short Texts Topic Modeling

STMLRC: Sparse Topic Model with Low Rank Constraint

FASTopic: A Fast, Adaptive, Stable, and Transferable Topic Modeling Paradigm

Scalable Topical Phrase Mining from Text Corpora

Topic model based on co-occurrence word networks for unbalanced short text datasets

Group Sparse Topical Coding

Topic Modeling in Semantic Space with Keywords.