Multimodal sentiment analysis of english and hinglish memes
Aishvi Guleria,Kamya Varshney,Garima Pahwa,Shreya Singhal,Nonita Sharma
DOI: https://doi.org/10.1007/s11042-024-19640-8
IF: 2.577
2024-06-21
Multimedia Tools and Applications
Abstract:Memes now-a-days have become a popular form of internet communication that individuals use to express complex emotions and viewpoints in an engaging yet concise manner. This research paper presents a groundbreaking study on the multimodal sentiment analysis of Hindi memes. Combining visual and linguistic components, the study investigates the intricate emotional context hidden within these culturally significant artifacts. A collection of 3999 Hinglish memes was assembled, a low-resource language, to conduct this analysis. The dataset combines verbal information and graphical elements, making it possible to analyze multimodal interactions thoroughly. Every picture has been marked according to the message communicated or the intended purpose and intention and is classified as "negative", "positive" or "neutral". The technical novelty of the study lies in the proposed multimodal approach for the sentiment analysis of Hinglish and English memes using a combination of RoBERTa and CLIP models, attaining an accuracy of 0.82 in contrast to the accuracy of 0.64 for the BERT model, 0.42 for the SVM model, 0.34 for multinomial NB model and 0.11 for VADER, which are some of the most widely used models for sentiment analysis. The findings highlight the significance of multimodal analysis in understanding and decoding the sentiment of Hinglish and English memes.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering