Abstract:Currently, social media has become an important means for people to share their daily lives and express emotions. Influenced by the overall environment, individuals are no longer limited to single-text content, but are increasingly inclined towards conveying information through various modalities. However, existing multimodal sentiment analysis methods face several challenges.Firstly, most current methods for sentiment polarity judgment primarily rely on textual content. These models struggle to effectively associate common emotional features between text and images. For instance, regardless of the textual content inputted into the model, the resulting image features remain the same for a given image. Thus, the models fail to extract relevant features from the text to aid in better sentiment analysis.Secondly, social media data commonly exhibit inconsistencies between textual content and image descriptions, making it difficult to obtain visually sensitive textual representations.Thirdly, most existing image feature extraction methods employ ResNet (Deep residual network) to directly fuse image features with text, lacking the ability to adaptively select and focus on key information from inputted image data.To address the aforementioned challenges in multimodal sentiment analysis, the following improvements have been made:1.Drawing on the target-oriented mBERT (TomBERT) model, which focuses on multimodal aspect-level sentiment analysis, we have designed a text-image matching module that is tailored to the experimental needs of this study. This module serves as the primary component applied in multimodal sentiment analysis.2.We have incorporated the Image Captioning with Transformers (CATR) module to facilitate the conversion of images into textual representations. This enriches the information available from the text data and resolves the issue of text-content mismatch with image descriptions.3.The improved Convolutional Block Attention Module (CBAM) was integrated to introduce the capability of adaptive selection and concentration on essential image features.In order to address the aforementioned issues, a model named Image Captioning Joint Attention Mechanism (ICAM) was designed.

A multimodal sentiment recognition method based on attention mechanism

Modality-invariant Temporal Representation Learning for Multimodal Sentiment Classification

Multimodal sentiment analysis based on multiple attention

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism

Image-text sentiment analysis via deep multimodal attentive fusion.

Context-Dependent Multimodal Sentiment Analysis Based on a Complex Attention Mechanism

Sentiment Analysis of Social Media Comments Based on Multimodal Attention Fusion Network

Multimodal Sentiment Analysis Based on Attentional Temporal Convolutional Network and Multi-Layer Feature Fusion

Feature Extraction Network with Attention Mechanism for Data Enhancement and Recombination Fusion for Multimodal Sentiment Analysis

A Multimodal Sentiment Analysis Method Integrating Multi-Layer Attention Interaction and Multi-Feature Enhancement

Social Image Sentiment Analysis by Exploiting Multimodal Content and Heterogeneous Relations

Multimodal sentiment analysis based on multi-head attention mechanism

Multi-Feature Fusion Multi-Modal Sentiment Analysis Model Based on Cross-Attention Mechanism

Multimodal Sentiment Recognition With Multi-Task Learning

Multimodal Sentiment Analysis Based on Image Captioning and Attention Mechanism

Social Media Public Opinion Detection Using Multimodal Natural Language Processing and Attention Mechanisms

A Multimodal Sentiment Analysis Approach Based on a Joint Chained Interactive Attention Mechanism

Multi-Modal Sentiment Analysis Based on Image and Text Fusion Based on Cross-Attention Mechanism

Sentiment Analysis Based on Text Information Enhancement and Multimodal Feature Fusion

A transformer-encoder-based multimodal multi-attention fusion network for sentiment analysis