Abstract:Topic modeling has been a key problem for document analysis. One of the canonical approaches for topic modeling is Probabilistic Latent Semantic Indexing, which maximizes the joint probability of documents and terms in the corpus. The major disadvantage of PLSI is that it estimates the probability distribution of each document on the hidden topics independently and the number of parameters in the model grows linearly with the size of the corpus, which leads to serious problems with overfitting. Latent Dirichlet Allocation (LDA) is proposed to overcome this problem by treating the probability distribution of each document over topics as a hidden random variable. Both of these two methods discover the hidden topics in the Euclidean space. However, there is no convincing evidence that the document space is Euclidean, or flat. Therefore, it is more natural and reasonable to assume that the document space is a manifold, either linear or nonlinear. In this paper, we consider the problem of topic modeling on intrinsic document manifold. Specifically, we propose a novel algorithm called Laplacian Probabilistic Latent Semantic Indexing (LapPLSI) for topic modeling. LapPLSI models the document space as a submanifold embedded in the ambient space and directly performs the topic modeling on this document manifold in question. We compare the proposed LapPLSI approach with PLSI and LDA on three text data sets. Experimental results show that LapPLSI provides better representation in the sense of semantic structure.

Joint Topic-Document Modeling Via Low-Dimensional Sparse Models

Document Clustering Based on Probabilistic Topic Model

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Incorporating Probabilistic Knowledge into Topic Models.

Incorporating Knowledge Graph Embeddings into Topic Modeling

A Novel Topic Model for Documents by Incorporating Semantic Relations Between Words

Parsimonious Topic Models with Salient Word Discovery

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Decomposing Background Topics from Keywords by Principal Component Pursuit.

A topic-based Document Correlation Model

Sparse Relational Topic Models for Document Networks.

Statistical Word Sense Aware Topic Models

STMLRC: Sparse Topic Model with Low Rank Constraint

Modeling Documents with Event Model

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey

Modeling Hidden Topics on Document Manifold

Topic Models Incorporating Statistical Word Senses

Topic-weak-correlated Latent Dirichlet Allocation

A density-based method for adaptive LDA model selection

Dirichlet Mixture Allocation for Multiclass Document Collections Modeling