Abstract:Topic modelling is a machine learning technique that is extensively used in Natural Language Processing (NLP) applications to infer topics within unstructured textual data. Latent Dirichlet Allocation (LDA) is one of the most used topic modeling techniques that can automatically detect topics from a huge collection of text documents. However, the LDA-based topic models alone do not always provide promising results. Clustering is one of the effective unsupervised machine learning algorithms that are extensively used in applications including extracting information from unstructured textual data and topic modeling. A hybrid model of Bidirectional Encoder Representations from Transformers (BERT) and Latent Dirichlet Allocation (LDA) in topic modeling with clustering based on dimensionality reduction have been studied in detail. As the clustering algorithms are computationally complex, the complexity increases with the higher number of features, the PCA, t-SNE and UMAP based dimensionality reduction methods are also performed. Finally, a unified clustering-based framework using BERT and LDA is proposed as part of this study for mining a set of meaningful topics from the massive text corpora. The experiments are conducted to demonstrate the effectiveness of the cluster-informed topic modeling framework using BERT and LDA by simulating user input on benchmark datasets. The experimental results show that clustering with dimensionality reduction would help infer more coherent topics and hence this unified clustering and BERT-LDA based approach can be effectively utilized for building topic modeling applications.

A self-supervised seed-driven approach to topic modelling and clustering

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts

A Process for Topic Modelling Via Word Embeddings

A Knowledge-Based Semisupervised Hierarchical Online Topic Detection Framework.

CAST: Corpus-Aware Self-similarity Enhanced Topic modelling

Topic Modeling Using Distributed Word Embeddings

Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms

Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words

A Cluster Guided Topic Model for Social Query Expansion

An Iterative Approach to Topic Modelling

Parsimonious Topic Models with Salient Word Discovery

Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds

Semantic Component Analysis: Discovering Patterns in Short Texts Beyond Topics

S2vNTM: Semi-supervised vMF Neural Topic Modeling

A semi-supervised approach of short text topic modeling using embedded fuzzy clustering for Twitter hashtag recommendation

Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings

An integrated clustering and BERT framework for improved topic modeling

Efficient and Flexible Topic Modeling using Pretrained Embeddings and Bag of Sentences

A Novel Topic Model for Documents by Incorporating Semantic Relations Between Words

Supervised cross-collection topic modeling.