Abstract:With the emergence and development of deep generative models, such as the variational auto-encoders (VAEs), the research on topic modeling successfully extends to a new area: neural topic modeling, which aims to learn disentangled topics to understand the data better. However, the original VAE framework had been shown to be limited in disentanglement performance, bringing their inherent defects to a neural topic model (NTM). In this paper, we put forward that the optimization objectives of contrastive learning are consistent with two important goals (alignment and uniformity) of well-disentangled topic learning. Also, the optimization objectives of contrastive learning are consistent with two key evaluation measures for topic models, topic coherence and topic diversity. So, we come to the important conclusion that alignment and uniformity of disentangled topic learning can be quantified with topic coherence and topic diversity. Accordingly, we are inspired to propose the Contrastive Disentangled Neural Topic Model (CNTM). By representing both words and topics as low-dimensional vectors in the same embedding space, we apply contrastive learning to neural topic modeling to produce factorized and disentangled topics in an interpretable manner. We compare our proposed CNTM with strong baseline models on widely-used metrics. Our model achieves the best topic coherence scores under the most general evaluation setting (100% proportion topic selected) with 25.0%, 10.9%, 24.6%, and 51.3% improvements above the second-best models’ scores reported on four datasets of 20 Newsgroups, Web Snippets, Tag My News, and Reuters, respectively. Our method also gets the second-best topic diversity scores on the dataset of 20Newsgroups and Web Snippets. Our experimental results show that CNTM can effectively leverage the disentanglement ability from contrastive learning to solve the inherent defect of neural topic modeling and obtain better topic quality.

Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion

Topic Evolution Analysis Based on Cluster Topic Model.

Contrastive Topic Evolution Discovery Via Nonnegative Matrix Factorization

Jointly Dynamic Topic Model for Recognition of Lead-lag Relationship in Two Text Corpora

Modeling the Evolution of Associated Data

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

Constructing Dynamic Topic Models Based on Variational Autoencoder and Factor Graph

FASTopic: A Fast, Adaptive, Stable, and Transferable Topic Modeling Paradigm

An integrated latent Dirichlet allocation and Word2vec method for generating the topic evolution of mental models from global to local

Adding Community and Dynamic to Topic Models.

Semantic-enhanced topic evolution analysis: a combination of the dynamic topic model and word2vec

Topic Modeling for Short Texts Via Dual View Collaborate Optimization

ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics

Improving Topic Disentanglement Via Contrastive Learning

A Framework for Semantic Connection Based Topic Evolution with DeepWalk

Neural Topic Modeling with Continual Lifelong Learning

Evolution Analysis of Online Topics Based on ‘Word-Topic’ Coupling Network

Identification of topic evolution: network analytics with piecewise linear representation and word embedding

Unraveling Scientific Evolutionary Paths: an Embedding-Based Topic Analysis

Continuous-time Infinite Dynamic Topic Models

A Dynamic Evolutionary Framework for Timeline Generation based on Distributed Representations