Abstract:Nonnegative matrix factorization (NMF) based topic modeling methods do not rely on model- or data-assumptions much. However, they are usually formulated as difficult optimization problems, which may suffer from bad local minima and high computational complexity. In this paper, we propose a deep NMF (DNMF) topic modeling framework to alleviate the aforementioned problems. It first applies an unsupervised deep learning method to learn latent hierarchical structures of documents, under the assumption that if we could learn a good representation of documents by, e.g. a deep model, then the topic word discovery problem can be boosted. Then, it takes the output of the deep model to constrain a topic-document distribution for the discovery of the discriminant topic words, which not only improves the efficacy but also reduces the computational complexity over conventional unsupervised NMF methods. We constrain the topic-document distribution in three ways, which takes the advantages of the three major sub-categories of NMF—basic NMF, structured NMF, and constrained NMF respectively. To overcome the weaknesses of deep neural networks in unsupervised topic modeling, we adopt a non-neural-network deep model—multilayer bootstrap network. To our knowledge, this is the first time that a deep NMF model is used for unsupervised topic modeling. We have compared the proposed method with a number of representative references covering major branches of topic modeling on a variety of real-world text corpora. Experimental results illustrate the effectiveness of the proposed method under various evaluation metrics.

Bridging Insight Gaps in Topic Dependency Discovery with a Knowledge-Inspired Topic Model

Short Text Understanding by Leveraging Knowledge into Topic Model.

Incorporating Knowledge Graph Embeddings into Topic Modeling

A Knowledge-Based Semisupervised Hierarchical Online Topic Detection Framework.

Hierarchical Topic Modeling with Nested Hierarchical Dirichlet Process

Interactive Topic Modeling Based on Hierarchical Dirichlet Process

Domain-specific Topic Model for Knowledge Discovery in Computational and Data-Intensive Scientific Communities

KDSTM: Neural Semi-supervised Topic Modeling with Knowledge Distillation

Hierarchical Topic Modeling with Automatic Knowledge Mining.

Topic Modeling Revisited: A Document Graph-based Neural Network Perspective

A Novel Topic Model for Documents by Incorporating Semantic Relations Between Words

On Modelling Non-Linear Topical Dependencies

Neural Topic Modeling with Deep Mutual Information Estimation

Incorporating Biterm Correlation Knowledge into Topic Modeling for Short Texts

Sentiment Analysis with Global Topics and Local Dependency

Keyword Assisted Embedded Topic Model

Topic Modeling over Short Texts by Incorporating Word Embeddings

Deep NMF topic modeling

On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling

A Novel Neural Topic Model and Its Supervised Extension.

KnowDT: Empathetic dialogue generation with knowledge enhanced dependency tree