Abstract:Deep probabilistic aspect models are widely utilized in document analysis to extract the semantic information and obtain descriptive topics. However, there are two problems that may affect their applications. One is that common words shared among all documents with low representational meaning may reduce the representation ability of learned topics. The other is introducing supervision information to hierarchical topic models to fully utilize the side information of documents that is difficult. To address these problems, in this article, we first propose deep diverse latent Dirichlet allocation (DDLDA), a deep hierarchical topic model that can yield more meaningful semantic topics with less common and meaningless words by introducing shared topics. Moreover, we develop a variational inference network for DDLDA, which helps us to further generalize DDLDA to a supervised deep topic model called max-margin DDLDA (mmDDLDA) by employing max-margin principle as the classification criterion. Compared to DDLDA, mmDDLDA can discover more discriminative topical representations. In addition, a continual hybrid method with stochastic-gradient MCMC and variational inference is put forward for deep latent Dirichlet allocation (DLDA)-based models to make them more practical in real-world applications. The experimental results demonstrate that DDLDA and mmDDLDA are more efficient than existing unsupervised and supervised topic models in discovering highly discriminative topic representations and achieving higher classification accuracy. Meanwhile, DLDA and our proposed models trained by the proposed continual learning approach cannot only show good performance on preventing catastrophic forgetting but also fit the evolving new tasks well.

Interpreting the Prediction Process of A Deep Network Constructed from Supervised Topic Models

Understanding The Prediction Process Of Deep Networks By Forests

End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture

Deep de Finetti: Recovering Topic Distributions from Large Language Models

Deep Latent Variable Predictive Modeling with Online Bayesian Soft Attention Mechanism

Deep Autoencoding Topic Model With Scalable Hybrid Bayesian Inference

Topic Analysis for Text with Side Data

Latent Concept-based Explanation of NLP Models

Deep PLS: A Lightweight Deep Learning Model for Interpretable and Efficient Data Analytics

Intepreting & Improving Pretrained Language Models: A Probabilistic Conceptual Approach

Explaining the Deep Natural Language Processing by Mining Textual Interpretable Features

Interpreting Deep Learning Models in Natural Language Processing: A Review

Exploring the effect of context information on deep learning business process predictions

A deep latent space model for interpretable representation learning on directed graphs

Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Towards Explanation of DNN-based Prediction with Guided Feature Inversion

A Unified Approach to Interpreting Model Predictions

Interpreting Deep Learning Model Using Rule-based Method

A Novel Business Process Prediction Model Using a Deep Learning Method

Max-Margin Deep Diverse Latent Dirichlet Allocation With Continual Learning