Abstract:Social network has become an inseparable part of our daily lives and thus the automatic sentiment analysis on social media content is of great significance to identify people’s viewpoints, attitudes, and emotions on the social websites. Most existing works have concentrated on the sentiment analysis of single modality such as image or text, which cannot handle the social media content with multiple modalities including both image and text. Although some works tried to conduct multi-modal sentiment analysis, the complicated correlations between the two modalities have not been fully explored. In this paper, we propose a novel Bi-Directional Multi-Level Attention (BDMLA) model to exploit the complementary and comprehensive information between the image modality and text modality for joint visual-textual sentiment classification. Specifically, to highlight the emotional regions and words in the image–text pair, visual attention network and semantic attention network are proposed respectively. The visual attention network makes region features of the image interact with multiple semantic levels of text (word, phrase, and sentence) to obtain the attended visual features. The semantic attention network makes semantic features of the text interact with multiple visual levels of image (global and local) to obtain the attended semantic features. Then, the attended visual and semantic features from the two attention networks are unified into a holistic framework to conduct visual-textual sentiment classification. Proof-of-concept experiments conducted on three real-world datasets verify the effectiveness of our model.

Parallel Multi-feature Attention on Neural Sentiment Classification

Improving Review Representations with User Attention and Product Attention for Sentiment Classification

Multi-Task Multi-Head Attention Memory Network for Fine-Grained Sentiment Analysis.

Neural Sentiment Classification with Social Feedback Signals.

Cascading Multiway Attentions for Document-level Sentiment Classification.

Neural Sentiment Classification with User and Product Attention

Hierarchical Neural Network with Serial Attention Mechanism for Review Sentiment Classifification

Modeling Different Effects of User and Product Attributes on Review Sentiment Classification

A HYBRID DEEP LEARNING APPROACH FOR SENTIMENT ANALYSIS IN PRODUCT REVIEWS

VisdaNet: Visual Distillation and Attention Network for Multimodal Sentiment Classification

Neural Classifier with Statistic Information of User and Product for Sentiment Analysis.

A Multi-sentiment-resource Enhanced Attention Network for Sentiment Classification

Sentiment Analysis With Comparison Enhanced Deep Neural Network

Sentiment Classification Via User and Product Interactive Modeling

Dual Memory Network Model for Sentiment Analysis of Review Text

Aspect-based Sentiment Analysis for Online Reviews with Hybrid Attention Networks

Visual-textual Sentiment Classification with Bi-Directional Multi-Level Attention Networks

Attention-based multi-level image and text sentiment analysis

Research on semantic orientation classification of chinese online product reviews based on multi-aspect sentiment analysis

Text Sentiment Classification Based on Layered Attention Network

Multi-Task Mutual Learning for Joint Sentiment Classification and Topic Detection