Abstract:Sentiment analysis of social media data is crucial to understand people’s position, attitude, and opinion toward a certain event, which has many applications such as election prediction and product evaluation. Though great effort has been devoted to the single modality (image or text), less effort is paid to the joint analysis of multimodal data in social media. Most of the existing methods for multimodal sentiment analysis simply combine different data modalities, which results in dissatisfying performance on sentiment classification. In this paper, we propose a novel image–text sentiment analysis model, i.e., Deep Multimodal Attentive Fusion (DMAF), to exploit the discriminative features and the internal correlation between visual and semantic contents with a mixed fusion framework for sentiment analysis. Specifically, to automatically focus on discriminative regions and important words which are most related to the sentiment, two separate unimodal attention models are proposed to learn effective emotion classifiers for visual and textual modality respectively. Then, an intermediate fusion-based multimodal attention model is proposed to exploit the internal correlation between visual and textual features for joint sentiment classification. Finally, a late fusion scheme is applied to combine the three attention models for sentiment prediction. Extensive experiments are conducted to demonstrate the effectiveness of our approach on both weakly labeled and manually labeled datasets.

Multi-attention Fusion Modeling for Sentiment Analysis of Educational Big Data

Sentiment Analysis Using Deep Robust Complementary Fusion of Multi-Features and Multi-Modalities.

Educational Big Data Analytics Using Sentiment Analysis for Student Requirement Analysis on Courses

Multi-modal Sentiment and Emotion Joint Analysis with a Deep Attentive Multi-task Learning Model

Multi-layer cross-modality attention fusion network for multimodal sentiment analysis

A cross modal hierarchical fusion multimodal sentiment analysis method based on multi-task learning

Multi-level Multi-task Representation Learning with Adaptive Fusion for Multimodal Sentiment Analysis

Multi-Channel Attentive Graph Convolutional Network with Sentiment Fusion for Multimodal Sentiment Analysis

Multimodal sentiment analysis based on multiple attention

Multi-Task Multi-Head Attention Memory Network for Fine-Grained Sentiment Analysis.

Multi-Feature Fusion Multi-Modal Sentiment Analysis Model Based on Cross-Attention Mechanism

M$^{3}$SA: Multimodal Sentiment Analysis Based on Multi-Scale Feature Extraction and Multi-Task Learning

Image-text sentiment analysis via deep multimodal attentive fusion.

Multi-Modal Sentiment Analysis Based on Image and Text Fusion Based on Cross-Attention Mechanism

Domain Attention Model for Multi-Domain Sentiment Classification.

Text Sentiment Analysis of Fusion Model Based on Attention Mechanism.

Multimodal Sentiment Analysis Using Multi-tensor Fusion Network with Cross-modal Modeling

A Multimodal Sentiment Analysis Method Integrating Multi-Layer Attention Interaction and Multi-Feature Enhancement

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism

Sentiment Analysis Methods Based on Multi-Modal Fusion and Deep Learning