Abstract:Today's Internet is awash in memes as they are humorous, satirical, or ironic which make people laugh. According to a survey, 33% of social media users in age bracket [13-35] send memes every day, whereas more than 50% send every week. Some of these memes spread rapidly within a very short time-frame, and their virality depends on the novelty of their (textual and visual) content. A few of them convey positive messages, such as funny or motivational quotes; while others are meant to mock/hurt someone's feelings through sarcastic or offensive messages. Despite the appealing nature of memes and their rapid emergence on social media, effective analysis of memes has not been adequately attempted to the extent it deserves. In this paper, we attempt to solve the same set of tasks suggested in the SemEval'20-Memotion Analysis competition. We propose a multi-hop attention-based deep neural network framework, called MHA-MEME, whose prime objective is to leverage the spatial-domain correspondence between the visual modality (an image) and various textual segments to extract fine-grained feature representations for classification. We evaluate MHA-MEME on the 'Memotion Analysis' dataset for all three sub-tasks - sentiment classification, affect classification, and affect class quantification. Our comparative study shows sota performances of MHA-MEME for all three tasks compared to the top systems that participated in the competition. Unlike all the baselines which perform inconsistently across all three tasks, MHA-MEME outperforms baselines in all the tasks on average. Moreover, we validate the generalization of MHA-MEME on another set of manually annotated test samples and observe it to be consistent. Finally, we establish the interpretability of MHA-MEME.

Meme Sentiment Analysis Enhanced with Multimodal Spatial Encoding and Facial Embedding

Unimodal Intermediate Training for Multimodal Meme Sentiment Classification

MemeFier: Dual-stage Modality Fusion for Image Meme Classification

Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis

Generating Multimodal Metaphorical Features for Meme Understanding

Research on Multimodal Sentiment Classification of Internet Memes Based on Transformer

Explainable Multimodal Sentiment Analysis on Bengali Memes

CEFM: CLIP Encoded Fusion Model for multimodal humor recognition on memes

A Multimodal Framework for the Detection of Hateful Memes

Emotion-Aware Multimodal Fusion for Meme Emotion Detection

Feels Bad Man: Dissecting Automated Hateful Meme Detection Through the Lens of Facebook's Challenge

Hateful Memes Detection via Complementary Visual and Linguistic Networks

Multimodal Deep Learning with Discriminant Descriptors for Offensive Memes Detection

Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification

Multimodal Analysis of memes for sentiment extraction

Multi-modal application: Image Memes Generation

Multimodal sentiment analysis of english and hinglish memes

Sentiment Extraction from Image-Based Memes Using Natural Language Processing and Machine Learning

MEMEX: Detecting Explanatory Evidence for Memes via Knowledge-Enriched Contextualization

Fuser: an Enhanced Multimodal Fusion Framework with Congruent Reinforced Perceptron for Hateful Memes Detection