Abstract:In a previous work, a parsimonious topic model (PTM) was proposed for text corpora. In that work, unlike LDA, the modeling determined a subset of salient words for each topic, with topic-specific probabilities, with the rest of the words in the dictionary explained by a universal shared model. Further, in LDA all topics are in principle present in every document. In contrast, PTM gives sparse topic representation, determining the (small) subset of relevant topics for each document. A customized Bayesian information criterion (BIC) was derived, balancing model complexity and goodness of fit, with the BIC minimized to jointly determine the entire model-the topic-specific words, document-specific topics, all model parameter values, and the total number of topics-in a wholly unsupervised fashion. In the present work, several important modeling and algorithm (parameter learning) extensions of PTM are proposed. First, we modify the BIC objective function using a lossless coding scheme with low modeling cost for describing words that are non-salient for all topics-such words are essentially identified as wholly noisy/uninformative. This approach increases the PTM's model sparsity, which also allows model selection of more topics and with lower BIC cost than the original PTM. Second, in the original PTM model learning strategy, word switches were updated sequentially, which is myopic and susceptible to finding poor locally optimal solutions. Here, instead, we jointly optimize all the switches that correspond to the same word (across topics). This approach jointly optimizes many more parameters at each step than the original PTM, which in principle should be less susceptible to finding poor local minima. Results on several document data sets show that our proposed method outperformed the original PTM model with respect to multiple performance measures, and gave a sparser topic model representation than the original PTM.

Topic Optimization Method Based on Pointwise Mutual Information

Topic Discovery Based on LDA_col Model and Topic Significance Re-ranking.

Incorporating Probabilistic Knowledge into Topic Models.

A density-based method for adaptive LDA model selection

Parsimonious Topic Models with Salient Word Discovery

A Method of Adaptively Selecting Best LDA Model Based on Density

Topic and Keyword Re-Ranking for LDA-based Topic Modeling

Improved Parsimonious Topic Modeling Based on the Bayesian Information Criterion

Method of adaptively selecting best LDA model based on density

Topic-weak-correlated Latent Dirichlet Allocation

Joint Topic-Document Modeling Via Low-Dimensional Sparse Models

MIT: Mutual Information Topic Model for Diverse Topic Extraction

Evaluating Topic Quality with Posterior Variability

Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings

LDAExplore: Visualizing Topic Models Generated Using Latent Dirichlet Allocation

Modeling Hidden Topics on Document Manifold

RankTopic: Ranking Based Topic Modeling

InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling

Improving the term weighting log entropy of latent dirichlet allocation

Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey

Novel mixture allocation models for topic learning