Abstract:Topic models have been widely used to learn text representations and gain insight into document corpora. To perform topic discovery, most existing neural models either take document bag-of-words (BoW) or sequence of tokens as input followed by variational inference and BoW reconstruction to learn topic-word distribution. However, leveraging topic-word distribution for learning better features during document encoding has not been explored much. To this end, we develop a framework TAN-NTM, which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. We propose a novel attention mechanism which factors in topic-word distribution to enable the model to attend on relevant words that convey topic related cues. The output of topic attention module is then used to carry out variational inference. We perform extensive ablations and experiments resulting in ~9-15 percentage improvement over score of existing SOTA topic models in NPMI coherence on several benchmark datasets - 20Newsgroups, Yelp Review Polarity and AGNews. Further, we show that our method learns better latent document-topic features compared to existing topic models through improvement on two downstream tasks: document classification and topic guided keyphrase generation.

Automatic Labelling of Topics with Neural Embeddings

Automatic Topic Labeling Using Graph-Based Pre-Trained Neural Embedding

Automatic Topic Labeling Model with Paired-Attention Based on Pre-trained Deep Neural Network.

Automatic Labelling Of Topic Models Using Word Vectors And Letter Trigram Vectors

Citation Metadata Extraction Via Deep Neural Network-based Segment Sequence Labeling

A Novel Topic Model for Automatic Term Extraction

Generative AI for automatic topic labelling

Automatic Labeling of Topic Models Using Text Summaries

Automatic Labelling Of Topic Models Learned From Twitter By Summarisation

Layer-Assisted Neural Topic Modeling over Document Networks

A Novel Neural Topic Model and Its Supervised Extension.

A Novel Sentence Embedding Based Topic Detection Method for Microblogs

Discovering topics with neural topic models built from PLSA assumptions

SAE-NTM: Sentence-Aware Encoder for Neural Topic Modeling

TAN-NTM: Topic Attention Networks for Neural Topic Modeling

Neural Topic Model with Distance Awareness

What Topics Do Images Say: A Neural Image Captioning Model with Topic Representation.

Automatic Labeling Of Topic Models Using Graph-Based Ranking

TAE: Topic-aware encoder for large-scale multi-label text classification

A Novel Sentence Embedding Based Topic Detection Method for Micro-Blog

Automatic Labeling Hierarchical Topics