Abstract:Millions of people post images and texts to express their feelings and point of views on social media everyday, especially on the short text social media such as Twitter or Weibo. As the images can provide important supplementary information for the text, many multimodal topic models have been developed to mine the topics from the multimodal social media content. We summarize three fundamental characteristics of the short text multimodal social media. The first is that the text of a short social media document generally belong to only one topic. The second is that the attached images can be relevant to multiple topics due to the rich information expressed in the images. The last is that although in most cases, text and images in social media posts are relevant, it should be noted that in a small number of cases, text and pictures are not relevant. However, most of the current multimodal topic models fail to model the these characteristics, and thus may produce low-quality topics. Based on these characteristics, we propose an unsupervised multimodal topic model SMMTM to model the short text multimodal social media documents. In the SMMTM model, only one topic is sampled for the the text while an image can belong to different topics. The correlation of the topics between the text and the images in a document are also formulated in an appropriate way. The experiments on three short text social media datasets with four evaluation metrics show the advantages of our model over the existing models.

Conceptualization Topic Modeling

Incorporating Knowledge Graph Embeddings into Topic Modeling

Bctm:A Topic Modeling Method Based on External Information

An Attention Hierarchical Topic Modeling

iTopicModel: Information Network-Integrated Topic Modeling

A Topic Model for Linked Documents and Update Rules for Its Estimation

Topic Modeling of Short Texts: A Pseudo-Document View with Word Embedding Enhancement.

Area Topic Model

Topic Modeling of Chinese Language Beyond a Bag-of-words

Topic Modeling based on Keywords and Context

Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings

Concept over Time: the Combination of Probabilistic Topic Model with Wikipedia Knowledge.

Multimodal Topic Modeling by Exploring Characteristics of Short Text Social Media

Latent Gaussian Models for Topic Modeling

Higher-Order Markov Tag-Topic Models for Tagged Documents and Images

A Hierarchical Latent Topic Model Based on Sparse Coding

Hierarchical Topic Integration Through Semi-Supervised Hierarchical Topic Modeling.

Incorporating Wikipedia Concepts and Categories As Prior Knowledge into Topic Models.

Nonparametric Bayesian Upstream Supervised Multi-Modal Topic Models

A Topic Model for Hierarchical Documents

A Hybrid Topic Model for Multi-Document Summarization