Abstract:Social media users articulate their opinions on a broad spectrum of subjects and share their experiences through posts comprising multiple modes of expression, leading to a notable surge in such multimodal content on social media platforms. Nonetheless, accurately forecasting the popularity of these posts presents a considerable challenge. Prevailing methodologies primarily center on the content itself, thereby overlooking the wealth of information encapsulated within alternative modalities such as visual demographics, sentiments conveyed through hashtags and adequately modeling the intricate relationships among hashtags, texts, and accompanying images. This oversight limits the ability to capture emotional connection and audience relevance, significantly influencing post popularity. To address these limitations, we propose a seNtiment and hAshtag-aware attentive deep neuRal netwoRk for multimodAl posT pOpularity pRediction, herein referred to as NARRATOR that extracts visual demographics from faces appearing in images and discerns sentiment from hashtag usage, providing a more comprehensive understanding of the factors influencing post popularity Moreover, we introduce a hashtag-guided attention mechanism that leverages hashtags as navigational cues, guiding the models focus toward the most pertinent features of textual and visual modalities, thus aligning with target audience interests and broader social media context. Experimental results demonstrate that NARRATOR outperforms existing methods by a significant margin on two real-world datasets. Furthermore, ablation studies underscore the efficacy of integrating visual demographics, sentiment analysis of hashtags, and hashtag-guided attention mechanisms in enhancing the performance of post popularity prediction, thereby facilitating increased audience relevance, emotional engagement, and aesthetic appeal.

Multimodal Deep Learning for Social Media Popularity Prediction With Attention Mechanism

A Feature Generalization Framework for Social Media Popularity Prediction

Automatic Depression Prediction Via Cross-Modal Attention-Based Multi-Modal Fusion in Social Networks

Combining Multiple Features for Image Popularity Prediction in Social Media.

Predicting Relative Popularity via an End-to-End Multi-modality Model.

An Attention-based Neural Model for Popularity Prediction in Social Service

Tri-Modal Transformers with Mixture-of-Modality-Experts for Social Media Prediction

Modeling and Predicting Popularity Dynamics via Deep Learning Attention Mechanism

Model Can Be Subtle: Two Important Mechanisms for Social Media Popularity Prediction

Contrastive Learning for Implicit Social Factors in Social Media Popularity Prediction

Improving Social Media Popularity Prediction with Multiple Post Dependencies

HyFea: Winning Solution to Social Media Popularity Prediction for Multimedia Grand Challenge 2020

User-guided Hierarchical Attention Network for Multi-modal Social Image Popularity Prediction.

A Real-Time Method to Predict Social Media Popularity

Sentiment and Hashtag-aware Attentive Deep Neural Network for Multimodal Post Popularity Prediction

SMP Challenge: An Overview and Analysis of Social Media Prediction Challenge

A multimodal sentiment recognition method based on attention mechanism

How to Become Instagram Famous: Post Popularity Prediction with Dual-Attention

Double-Fine-Tuning Multi-Objective Vision-and-Language Transformer for Social Media Popularity Prediction

On the Limits to Multi-Modal Popularity Prediction on Instagram -- A New Robust, Efficient and Explainable Baseline

Multi‐Pop: Enhancing user engagement with content‐based multimodal popularity prediction in social media