Abstract:Deep probabilistic aspect models are widely utilized in document analysis to extract the semantic information and obtain descriptive topics. However, there are two problems that may affect their applications. One is that common words shared among all documents with low representational meaning may reduce the representation ability of learned topics. The other is introducing supervision information to hierarchical topic models to fully utilize the side information of documents that is difficult. To address these problems, in this article, we first propose deep diverse latent Dirichlet allocation (DDLDA), a deep hierarchical topic model that can yield more meaningful semantic topics with less common and meaningless words by introducing shared topics. Moreover, we develop a variational inference network for DDLDA, which helps us to further generalize DDLDA to a supervised deep topic model called max-margin DDLDA (mmDDLDA) by employing max-margin principle as the classification criterion. Compared to DDLDA, mmDDLDA can discover more discriminative topical representations. In addition, a continual hybrid method with stochastic-gradient MCMC and variational inference is put forward for deep latent Dirichlet allocation (DLDA)-based models to make them more practical in real-world applications. The experimental results demonstrate that DDLDA and mmDDLDA are more efficient than existing unsupervised and supervised topic models in discovering highly discriminative topic representations and achieving higher classification accuracy. Meanwhile, DLDA and our proposed models trained by the proposed continual learning approach cannot only show good performance on preventing catastrophic forgetting but also fit the evolving new tasks well.

Variable Selection for Latent Dirichlet Allocation

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

Probabilistic Word Selection Via Topic Modeling

A density-based method for adaptive LDA model selection

Variable Selection in Discriminant Partial Least-Squares Analysis

A Spectral Algorithm for Latent Dirichlet Allocation

Variational Discriminant Analysis with Variable Selection

Novel mixture allocation models for topic learning

Variable Selection Using Nonlocal Priors in High-Dimensional Generalized Linear Models With Application to fMRI Data Analysis

A Novel Variational Bayesian Method for Variable Selection in Logistic Regression Models.

Topic-weak-correlated Latent Dirichlet Allocation

ELBD: Efficient score algorithm for feature selection on latent variables of VAE

Variable selection for partially linear models via Bayesian subset modeling with diffusing prior

Dirichlet Mixture Allocation for Multiclass Document Collections Modeling

Variable Selection in High-Dimensional Error-in-Variables Models Via Controlling the False Discovery Proportion

Max-Margin Deep Diverse Latent Dirichlet Allocation With Continual Learning

LogisticLDA: Regularizing Latent Dirichlet Allocation by Logistic Regression

Spectral Learning for Supervised Topic Models

Variable selection in model-based clustering and discriminant analysis with a regularization approach

Study of Bayesian variable selection method on mixed linear regression models

Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey