Abstract:Topic model is a practical method for learning interpretable models of text corpora and have become a key problem of document representation. Some recently proposed topic models incorporate the intrinsic geometrical information of the document manifold and yield a discriminative topic representation. However, the existing manifold-inspired topic models fail to provide the probability weighting information of local geometrical pattern, thus leads to a limitation to estimate intrinsic semantic information of topic representation. In this paper, we consider the problem of topic modeling with intrinsic structure of document manifold and propose an unsupervised AutoEncoder-based topic modeling framework, named locally weighted embedding topic model (LWE-TM). Different from existing manifold-inspired topic models, LWE-TM defines a group of probability coefficients to uncover the local geometrical pattern by the Markov random walk structure of affinity graph, and regularizes the training of sparse AutoEncoder (sAE) to explicitly recover such local geometrical pattern with the topics encoding. Under the regularized training framework, the encoding network becomes local-invariant around the neighborhood of the document manifold and enable us to perform a readily topic inference for out-of-sample documents, efficiently improving the generalization and discrimination of topics encoding. The experimental results on two widely-used corpus demonstrate the superiority of LWE-TM to comparative models in document modeling, document clustering and classification tasks.

Topic Modeling in Embedding Spaces

Topic Modeling over Short Texts by Incorporating Word Embeddings

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Incorporating Knowledge Graph Embeddings into Topic Modeling

A Weighted Topic Modeling Approach Based on Word Embedding

Topic Modeling Using Distributed Word Embeddings

Keyword Assisted Embedded Topic Model

Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings

Topic Model over Short Texts Incorporating Word Embedding

Nested Variational Autoencoder for Topic Modeling on Microtexts with Word Vectors

Exclusive Topic Modeling

Integrating Topic Modeling with Word Embeddings by Mixtures of Vmfs.

Locally Weighted Embedding Topic Modeling by Markov Random Walk Structure Approximation and Sparse Regularization

Topic Discovery by Spectral Decomposition and Clustering with Coordinated Global and Local Contexts

A Correlated Topic Model Using Word Embeddings

DiffETM: Diffusion Process Enhanced Embedded Topic Model

Topic Modeling of Short Texts: A Pseudo-Document View with Word Embedding Enhancement.

Financial Topic Modeling Based on the BERT-LDA Embedding

Topic2Vec: Learning distributed representations of topics

Topic Discovery for Short Texts Using Word Embeddings

Leveraging external information in topic modelling