Abstract:Currently, social media has become an important means for people to share their daily lives and express emotions. Influenced by the overall environment, individuals are no longer limited to single-text content, but are increasingly inclined towards conveying information through various modalities. However, existing multimodal sentiment analysis methods face several challenges.Firstly, most current methods for sentiment polarity judgment primarily rely on textual content. These models struggle to effectively associate common emotional features between text and images. For instance, regardless of the textual content inputted into the model, the resulting image features remain the same for a given image. Thus, the models fail to extract relevant features from the text to aid in better sentiment analysis.Secondly, social media data commonly exhibit inconsistencies between textual content and image descriptions, making it difficult to obtain visually sensitive textual representations.Thirdly, most existing image feature extraction methods employ ResNet (Deep residual network) to directly fuse image features with text, lacking the ability to adaptively select and focus on key information from inputted image data.To address the aforementioned challenges in multimodal sentiment analysis, the following improvements have been made:1.Drawing on the target-oriented mBERT (TomBERT) model, which focuses on multimodal aspect-level sentiment analysis, we have designed a text-image matching module that is tailored to the experimental needs of this study. This module serves as the primary component applied in multimodal sentiment analysis.2.We have incorporated the Image Captioning with Transformers (CATR) module to facilitate the conversion of images into textual representations. This enriches the information available from the text data and resolves the issue of text-content mismatch with image descriptions.3.The improved Convolutional Block Attention Module (CBAM) was integrated to introduce the capability of adaptive selection and concentration on essential image features.In order to address the aforementioned issues, a model named Image Captioning Joint Attention Mechanism (ICAM) was designed.

Bidirectional Complementary Correlation-Based Multimodal Aspect-Level Sentiment Analysis

Dual Edge-embedding Graph Convolutional Network for Unified Aspect-based Sentiment Analysis

Image and Text Aspect Level Multimodal Sentiment Classification Model Using Transformer and Multilayer Attention Interaction

An Interactive Attention Mechanism Fusion Network for Aspect-Based Multimodal Sentiment Analysis

Exploring Multimodal Sentiment Analysis via CBAM Attention and Double-layer BiLSTM Architecture

Target-oriented Sentiment Classification with Sequential Cross-modal Semantic Graph

Dual-Perspective Fusion Network for Aspect-Based Multimodal Sentiment Analysis

Research on Multimodal Aspect-Based Sentiment Analysis Based on Image Caption and Multimodal Aspect Extraction

MIECF: Multi-faceted information extraction and cross-mixture fusion for multimodal aspect-based sentiment analysis

Cross-Modal Multitask Transformer for End-to-End Multimodal Aspect-Based Sentiment Analysis

MSFNet: modality smoothing fusion network for multimodal aspect-based sentiment analysis

Short Text Semantic Sentiment Analysis Based on Dual Channel Aspect Attention in Intelligent Systems

Interactive Fusion Network with Recurrent Attention for Multimodal Aspect-based Sentiment Analysis.

Multimodal Sentiment Analysis Based on Image Captioning and Attention Mechanism

ModalNet: an aspect-level sentiment classification model by exploring multimodal data with fusion discriminant attentional network

Cross-Modal Sentiment Analysis of Text and Video Based on Bi-GRU Cyclic Network and Correlation Enhancement

Multifeature Interactive Fusion Model for Aspect-Based Sentiment Analysis

AoM: Detecting Aspect-oriented Information for Multimodal Aspect-Based Sentiment Analysis

Multi-selection Attention for Multimodal Aspect-level Sentiment Classification

Image-to-Text Conversion and Aspect-Oriented Filtration for Multimodal Aspect-Based Sentiment Analysis

Aspects Are Anchors: Towards Multimodal Aspect-based Sentiment Analysis Via Aspect-driven Alignment and Refinement