Abstract:We propose selective supervised Latent Dirichlet Allocation (ssLDA) to boost the prediction performance of the widely studied supervised probabilistic topic models. We introduce a Bernoulli distribution for each word in one given document to select this word as a strongly or weakly discriminative one with respect to its assigned topic. The Bernoulli distribution is parameterized by the discrimination power of the word for its assigned topic. As a result, the document is represented as a "bag-of-selective-words" instead of the probabilistic "bag-of-topics" in the topic modeling domain or the flat "bag-of-words" in the traditional natural language processing domain to form a new perspective. Inheriting the general framework of supervised LDA (sLDA), ssLDA can also predict many types of response specified by a Gaussian Linear Model (GLM). Focusing on the utilization of this word selection mechanism for singe-label document classification in this paper, we conduct the variational inference for approximating the intractable posterior and derive a maximum-likelihood estimation of parameters in ssLDA. The experiments reported on textual documents show that ssLDA not only performs competitively over "state-of-the-art" classification approaches based on both the flat "bag-of-words" and probabilistic "bag-of-topics" representation in terms of classification performance, but also has the ability to discover the discrimination power of the words specified in the topics (compatible with our rational knowledge).

A density-based method for adaptive LDA model selection

Method of adaptively selecting best LDA model based on density

A Method of Adaptively Selecting Best LDA Model Based on Density

Topic Discovery Based on LDA_col Model and Topic Significance Re-ranking.

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Topic Selection in Latent Dirichlet Allocation.

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

Unsupervised feature selection for Latent Dirichlet Allocation

Probabilistic Word Selection Via Topic Modeling

Variable Selection for Latent Dirichlet Allocation

A Spectral Algorithm for Latent Dirichlet Allocation

Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey

Supervised Latent Dirichlet Allocation with a Mixture of Sparse Softmax

Topic-weak-correlated Latent Dirichlet Allocation

DiscLDA: Discriminative learning for dimensionality reduction and classification

Dirichlet Mixture Allocation for Multiclass Document Collections Modeling

Exploring Topic Discriminating Power of Words in Latent Dirichlet Allocation.

LogisticLDA: Regularizing Latent Dirichlet Allocation by Logistic Regression

Efficient Methods for Incorporating Knowledge into Topic Models

Multi-LDA hybrid topic model with boosting strategy and its application in text classification

Latent Dirichlet Allocation - An approach for topic discovery