Abstract:Nonnegative matrix factorization (NMF) based topic modeling methods do not rely on model- or data-assumptions much. However, they are usually formulated as difficult optimization problems, which may suffer from bad local minima and high computational complexity. In this paper, we propose a deep NMF (DNMF) topic modeling framework to alleviate the aforementioned problems. It first applies an unsupervised deep learning method to learn latent hierarchical structures of documents, under the assumption that if we could learn a good representation of documents by, e.g. a deep model, then the topic word discovery problem can be boosted. Then, it takes the output of the deep model to constrain a topic-document distribution for the discovery of the discriminant topic words, which not only improves the efficacy but also reduces the computational complexity over conventional unsupervised NMF methods. We constrain the topic-document distribution in three ways, which takes the advantages of the three major sub-categories of NMF—basic NMF, structured NMF, and constrained NMF respectively. To overcome the weaknesses of deep neural networks in unsupervised topic modeling, we adopt a non-neural-network deep model—multilayer bootstrap network. To our knowledge, this is the first time that a deep NMF model is used for unsupervised topic modeling. We have compared the proposed method with a number of representative references covering major branches of topic modeling on a variety of real-world text corpora. Experimental results illustrate the effectiveness of the proposed method under various evaluation metrics.

Topic Modeling with Network Regularization

GSLDA: Supervised topic model with graph regularization

Effective Neural Topic Modeling with Embedding Clustering Regularization

Topic Modeling Revisited: A Document Graph-based Neural Network Perspective

Twitter-Network Topic Model: A Full Bayesian Treatment for Social Network and Text Modeling

A network approach to topic models

Discriminative Relational Topic Models.

Incorporating Social Role Theory into Topic Models for Social Media Content Analysis.

A Novel Neural Topic Model and Its Supervised Extension.

Neural Topic Modeling by Incorporating Document Relationship Graph

Community-based Topic Modeling for Social Tagging.

Adding Community and Dynamic to Topic Models.

Graph Structural-topic Neural Network

Generalized Relational Topic Models with Data Augmentation.

Sparse Relational Topic Models for Document Networks.

Towards Better Understanding with Uniformity and Explicit Regularization of Embeddings in Embedding-based Neural Topic Models

TAN-NTM: Topic Attention Networks for Neural Topic Modeling

Semi-supervised topic modeling for image annotation.

Enhancing Topic Interpretability for Neural Topic Modeling Through Topic-Wise Contrastive Learning

Deep NMF topic modeling

Collaborative Topic Regression with Social Regularization for Tag Recommendation.