Abstract:In the circumstance of social big data, sentiment analysis is attracting increasing attention for its capacity in understanding individuals' attitudes and feelings. Traditional sentiment analysis methods focus on single modality and become ineffective as enormous data are emerging on the social websites with multiple manifestations. In this article, multimodal learning approaches are proposed to capture the relations between image and text, which only stay at the region level and ignore the fact that the channels are also closely correlated with the semantic information. In addition, social images in the social platforms are closely connected by various types of relations, which are also conducice to sentiment classification but neglected by most existing works. In this article, we propose an attention-based heterogeneous relational model to improve the multimodal sentiment analysis performance by incorporating rich social information. Specifically, we propose a progressive dual attention module to capture the correlations between image and text, and then learn the joint image-text representation from the perspective of content information. A channel attention schema is proposed here to highlight semantically rich image channels and a region attention schema is further designed to highlight the emotional regions based on the attended channels. After that, we construct a heterogeneous relation network and extend graph convolutional network to aggregate the content information from social contexts as complements to learn high-quality representations of social images. Our proposal is thoroughly evaluated on two benchmark datasets, and experimental results demonstrate the superiority of the proposed model.

A Sentiment Analysis Method for Big Social Online Multimodal Comments Based on Pre-trained Models

A Visual Approach to Tracking Emotional Sentiment Dynamics in Social Network Commentaries

SEntiMoji: An Emoji-Powered Learning Approach for Sentiment Analysis in Software Engineering

Emotion Analysis Model of Microblog Comment Text Based on CNN-BiLSTM

Performance evaluation of Reddit Comments using Machine Learning and Natural Language Processing methods in Sentiment Analysis

Topic-Specific Emotion Mining Model for Online Comments

Multimodal sentiment analysis leveraging the strength of deep neural networks enhanced by the XGBoost classifier

Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages

MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms

An Experimental Analysis of Data Annotation Methodologies for Emotion Detection in Short Text Posted on Social Media

Multimodal Sentiment Analysis of Government Information Comments Based on Contrastive Learning and Cross-Attention Fusion Networks

Performing Sentiment Analysis with the Proposed Artificial Intelligence-Based Method Using Movie Commentaries

Social Image Sentiment Analysis by Exploiting Multimodal Content and Heterogeneous Relations

Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning

An efficient multimodal sentiment analysis in social media using hybrid optimal multi-scale residual attention network

Multimodal Sentiment Sensing and Emotion Recognition Based on Cognitive Computing Using Hidden Markov Model with Extreme Learning Machine

EMFSA: Emoji-based multifeature fusion sentiment analysis

Scalable deep learning framework for sentiment analysis prediction for online movie reviews

Sentiment analysis of online public opinion based on CNN-BiLSTM and attention mechanism

Transformer Based Memory Network for Sentiment Analysis of Web Comments.

Comparison of Machine Learning Approaches for Sentiment Analysis in Slovak