Abstract:We propose selective supervised Latent Dirichlet Allocation (ssLDA) to boost the prediction performance of the widely studied supervised probabilistic topic models. We introduce a Bernoulli distribution for each word in one given document to select this word as a strongly or weakly discriminative one with respect to its assigned topic. The Bernoulli distribution is parameterized by the discrimination power of the word for its assigned topic. As a result, the document is represented as a "bag-of-selective-words" instead of the probabilistic "bag-of-topics" in the topic modeling domain or the flat "bag-of-words" in the traditional natural language processing domain to form a new perspective. Inheriting the general framework of supervised LDA (sLDA), ssLDA can also predict many types of response specified by a Gaussian Linear Model (GLM). Focusing on the utilization of this word selection mechanism for singe-label document classification in this paper, we conduct the variational inference for approximating the intractable posterior and derive a maximum-likelihood estimation of parameters in ssLDA. The experiments reported on textual documents show that ssLDA not only performs competitively over "state-of-the-art" classification approaches based on both the flat "bag-of-words" and probabilistic "bag-of-topics" representation in terms of classification performance, but also has the ability to discover the discrimination power of the words specified in the topics (compatible with our rational knowledge).

Topic Selection in Latent Dirichlet Allocation.

A density-based method for adaptive LDA model selection

Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey

Latent Dirichlet Allocation - An approach for topic discovery

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

Unsupervised feature selection for Latent Dirichlet Allocation

Latent dirichlet allocation

Method of adaptively selecting best LDA model based on density

Performance evaluation of Latent Dirichlet Allocation in text mining.

A Method of Adaptively Selecting Best LDA Model Based on Density

Exploring Topic Discriminating Power of Words in Latent Dirichlet Allocation.

Probabilistic Word Selection Via Topic Modeling

Variable Selection for Latent Dirichlet Allocation

Dirichlet Mixture Allocation for Multiclass Document Collections Modeling

Topic-weak-correlated Latent Dirichlet Allocation

Topic Mining for Call Centers Based on LDA

LDAExplore: Visualizing Topic Models Generated Using Latent Dirichlet Allocation

A Spectral Algorithm for Latent Dirichlet Allocation

A LDA Model Based Topic Detection Method

Research Topic Recommendation Based on Latent Dirichlet Allocation

Text Similarity Computing Based on Topic Model LDA