Abstract:Topic models have shown great promise in discovering latent semantic structures from complex data corpora, ranging from text documents and web news articles to images, videos, and even biological data. In order to deal with massive data collections and dynamic text streams, probabilistic online topic models such as online latent Dirichlet allocation (OLDA) have recently been developed. However, due to normalization constraints, OLDA can be ineffective in controlling the sparsity of discovered representations, a desirable property for learning interpretable semantic patterns, especially when the total number of topics is large. In contrast, sparse topical coding (STC) has been successfully introduced as a non-probabilistic topic model for effectively discovering sparse latent patterns by using sparsity-inducing regularization. But, unfortunately STC cannot scale to very large datasets or deal with online text streams, partly due to its batch learning procedure. In this paper, we present a sparse online topic model, which directly controls the sparsity of latent semantic patterns by imposing sparsity-inducing regularization and learns the topical dictionary by an online algorithm. The online algorithm is efficient and guaranteed to converge. Extensive empirical results of the sparse online topic model as well as its collapsed and supervised extensions on a large-scale Wikipedia dataset and the medium-sized 20Newsgroups dataset demonstrate appealing performance.

Sparse online topic models

Short Text Understanding by Leveraging Knowledge into Topic Model.

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

STMLRC: Sparse Topic Model with Low Rank Constraint

Group Sparse Topical Coding

Parsimonious Topic Models with Salient Word Discovery

Sparse Relational Topic Models for Document Networks.

Online Subset Topic Modeling For Interactive Documents Exploration

A Hierarchical Latent Topic Model Based on Sparse Coding

Conditional topical coding: an efficient topic model conditioned on rich features.

Topic Modeling in Semantic Space with Keywords.

Topic-weak-correlated Latent Dirichlet Allocation

Efficient Topic Modeling on Phrases via Sparsity

A Topic Model for Co-Occurring Normal Documents and Short Texts.

Sparse Relational Topical Coding on Multi-Modal Data.

Sparseness-constrained Nonnegative Tensor Factorization for Detecting Topics at Different Time Scales

Joint Latent Dirichlet Allocation for Non-Iid Social Tags.

Efficient Methods for Incorporating Knowledge into Topic Models

Locally discriminative topic modeling

Topic model based on co-occurrence word networks for unbalanced short text datasets