Abstract:With the emergence and development of deep generative models, such as the variational auto-encoders (VAEs), the research on topic modeling successfully extends to a new area: neural topic modeling, which aims to learn disentangled topics to understand the data better. However, the original VAE framework had been shown to be limited in disentanglement performance, bringing their inherent defects to a neural topic model (NTM). In this paper, we put forward that the optimization objectives of contrastive learning are consistent with two important goals (alignment and uniformity) of well-disentangled topic learning. Also, the optimization objectives of contrastive learning are consistent with two key evaluation measures for topic models, topic coherence and topic diversity. So, we come to the important conclusion that alignment and uniformity of disentangled topic learning can be quantified with topic coherence and topic diversity. Accordingly, we are inspired to propose the Contrastive Disentangled Neural Topic Model (CNTM). By representing both words and topics as low-dimensional vectors in the same embedding space, we apply contrastive learning to neural topic modeling to produce factorized and disentangled topics in an interpretable manner. We compare our proposed CNTM with strong baseline models on widely-used metrics. Our model achieves the best topic coherence scores under the most general evaluation setting (100% proportion topic selected) with 25.0%, 10.9%, 24.6%, and 51.3% improvements above the second-best models’ scores reported on four datasets of 20 Newsgroups, Web Snippets, Tag My News, and Reuters, respectively. Our method also gets the second-best topic diversity scores on the dataset of 20Newsgroups and Web Snippets. Our experimental results show that CNTM can effectively leverage the disentanglement ability from contrastive learning to solve the inherent defect of neural topic modeling and obtain better topic quality.

An optimized BERTopic framework based on cluster silhouette for improving topic coherence

Enhancing sentence-level clustering with integrated and interactive frameworks for theme-based summarization

An integrated clustering and BERT framework for improved topic modeling

Enhancing sentence-level clustering with ranking-based clustering framework for theme-based summarization

Topic Modeling for Short Texts Via Dual View Collaborate Optimization

Enhancing diversity and coverage of document summaries through subspace clustering and clustering-based optimization

Topic Discovery by Spectral Decomposition and Clustering with Coordinated Global and Local Contexts

Topic Evolution Analysis Based on Cluster Topic Model.

A Cluster Guided Topic Model for Social Query Expansion

Parsimonious Topic Models with Salient Word Discovery

Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling

An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features

Hybrid Self-Optimized Clustering Model Based on Citation Links and Textual Features to Detect Research Topics.

A New Method of Cluster-Based Topic Language Model for Genomic IR

Incorporating Biterm Correlation Knowledge into Topic Modeling for Short Texts

Optimising Topic Coherence with Weighted Po ´lya Urn Scheme

A topic-enhanced dirichlet model for short text stream clustering

LIMTopic: A Framework of Incorporating Link Based Importance into Topic Modeling

Improving Search Result Clustering by Enriching Snippets with Word2Vec Model.

Exploring coherent topics by topic modeling with term weighting.

Improving Topic Disentanglement Via Contrastive Learning