Abstract:Topic modeling has been a key problem for document analysis. One of the canonical approaches for topic modeling is Probabilistic Latent Semantic Indexing, which maximizes the joint probability of documents and terms in the corpus. The major disadvantage of PLSI is that it estimates the probability distribution of each document on the hidden topics independently and the number of parameters in the model grows linearly with the size of the corpus, which leads to serious problems with overfitting. Latent Dirichlet Allocation (LDA) is proposed to overcome this problem by treating the probability distribution of each document over topics as a hidden random variable. Both of these two methods discover the hidden topics in the Euclidean space. However, there is no convincing evidence that the document space is Euclidean, or flat. Therefore, it is more natural and reasonable to assume that the document space is a manifold, either linear or nonlinear. In this paper, we consider the problem of topic modeling on intrinsic document manifold. Specifically, we propose a novel algorithm called Laplacian Probabilistic Latent Semantic Indexing (LapPLSI) for topic modeling. LapPLSI models the document space as a submanifold embedded in the ambient space and directly performs the topic modeling on this document manifold in question. We compare the proposed LapPLSI approach with PLSI and LDA on three text data sets. Experimental results show that LapPLSI provides better representation in the sense of semantic structure.

Content Modeling Using Latent Permutations

Hierarchical Topic Modeling with Nested Hierarchical Dirichlet Process

Parsimonious Topic Models with Salient Word Discovery

Learning Topic Hierarchies by Tree-Directed Latent Variable Models

Jointly Modeling Topics and Intents with Global Order Structure.

Latent Gaussian Models for Topic Modeling

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

A Latent Topic Model for Linked Documents

Modeling Hidden Topics on Document Manifold

A Topic Model for Linked Documents and Update Rules for Its Estimation

Topic-based mixture language modelling

A Hierarchical Latent Topic Model Based on Sparse Coding

Graph-Structured Topic Modeling for Documents with Spatial or Covariate Dependencies

Gaussian Hierarchical Latent Dirichlet Allocation: Bringing Polysemy Back

Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings

Group Topic Model: Organizing Topics into Groups.

Deep de Finetti: Recovering Topic Distributions from Large Language Models

Dirichlet Multinomial Mixture with Variational Manifold Regularization: Topic Modeling over Short Texts

Conceptualization Topic Modeling

An Attention Hierarchical Topic Modeling

Labeled Phrase Latent Dirichlet Allocation