Abstract:In recent years, multimodal topic models have gained significant attention in various tasks involving short texts. Despite their impressive results, most models rely on bag-of-words assumptions for each modality, neglecting the intrinsic word relations in the textual modality and the underlying object relations in the visual modality. To address this limitation, we propose a novel approach that represents each document modality as a graph, harnessing the word relations and the visual object relations to guide the topic extraction process. Our approach is grounded in the insight that, in the textual modality, words with specific relations, such as co-occurrence relations, semantic relations and syntactic relations, are more likely to be assigned to the same topic. Similarly, in the visual modality, the relations between objects, such as spatial relations and contextual relations, can also provide valuable information for topic extraction. By leveraging graph-based representations, our model captures the inherent associations between words and visual objects, resulting in the generation of more coherent and interpretable topics. To infer the model's parameters, we develop an effective algorithm that integrates neural variational inference and contrastive learning. The experimental results on three datasets verify the effectiveness of our proposed model in terms of topic coherence, topic diversity and mean average precision, confirming that incorporating word relations and object relations through graph-based representations significantly enhances the quality of the extracted topics.

Describing Web Topics Meticulously through Word Graph Analysis

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Exploring simultaneous keyword and key sentence extraction: improve graph-based ranking using wikipedia.

Automatic Keyphrase Extraction Via Topic Decomposition.

Topic Analysis by Detecting Keywords

Intensity of Relationship Between Words: Using Word Triangles in Topic Discovery for Short Texts

Interactive Visual Exploration of Topic Models using Graphs

Web Forum Sentiment Analysis Based on Topics

Annotation-aware web clustering based on topic model and random walks

Topic Modeling Based Text Summarization Approach

WordTopic-MultiRank: A New Method for Automatic Keyphrase Extraction.

Analysis of the Research Topics of Webometrics——Based on Co-word Visualization

Graph-based Multimodal Topic Modeling with Word Relations and Object Relations

"Draw My Topics": Find Desired Topics fast from large scale of Corpus

Optimizing temporal topic segmentation for intelligent text visualization.

Internet Popular Topics Extraction of Traffic Content Words Correlation

Social Context Summarization

Using Big Data From The Web To Train Chinese Traffic Word Representation Model In Vector Space

Topic Analysis for Topic-Focused Multi-Document Summarization

Topic-based Visual Text Summarization and Analysis 1

Web video topic discovery and tracking via bipartite graph reinforcement model.