Abstract:Aspect-level sentiment analysis within multimodal contexts, focusing on the precise identification and interpretation of sentiment attitudes linked to the target aspect across diverse data modalities, remains a focal research area that perpetuates the advancement of discourse and innovation in artificial intelligence. However, most existing methods tend to focus on extracting visual features from only one facet, such as face expression, which ignores the value of information from other key facets, such as the textual information presented by the image modality, resulting in information loss. To overcome the aforementioned constraint, we put forth a novel approach designated as Multi-faceted Information Extraction and Cross-mixture Fusion (MIECF) for Multimodal Aspect-based Sentiment Analysis. Our approach captures more comprehensive visual information in the image and integrates these local and global key features from multiple facets. Local features, such as facial expressions and textual features, provide direct and rich emotional cues. By contrast, the global feature often reflects the overall emotional atmosphere and context. To enhance the visual representation, we designed a Cross-mixture Fusion method to integrate this local and global multimodal information. In particular, the method establishes semantic relationships between local and global features to eliminate ambiguity brought by single-facet information and achieve more accurate contextual understanding, providing a richer and more precise manner for sentiment analysis. The experimental findings indicate that our proposed approach achieves a leading level of performance, resulting in an Accuracy of 79.65 % on the Twitter-2015 dataset, and Macro-F1 scores of 75.90 % and 73.11 % for the Twitter-2015 and Twitter-2017 datasets, respectively.

Cross-modal fine-grained alignment and fusion network for multimodal aspect-based sentiment analysis

Multi-level textual-visual alignment and fusion network for multimodal aspect-based sentiment analysis

MSFNet: modality smoothing fusion network for multimodal aspect-based sentiment analysis

A cross-model hierarchical interactive fusion network for end-to-end multimodal aspect-based sentiment analysis

Cross-modal Contrastive Learning for Multimodal Fake News Detection

Cross-modal Enhancement Network for Multimodal Sentiment Analysis

Multi-layer cross-modality attention fusion network for multimodal sentiment analysis

MIECF: Multi-faceted information extraction and cross-mixture fusion for multimodal aspect-based sentiment analysis

CiteNet: Cross-modal incongruity perception network for multimodal sentiment prediction

Hierarchical Fusion Network with Enhanced Knowledge and Contrastive Learning for Multimodal Aspect-Based Sentiment Analysis on Social Media

ModalNet: an aspect-level sentiment classification model by exploring multimodal data with fusion discriminant attentional network

CTHFNet: contrastive translation and hierarchical fusion network for text–video–audio sentiment analysis

An Interactive Attention Mechanism Fusion Network for Aspect-Based Multimodal Sentiment Analysis

Multi-Channel Attentive Graph Convolutional Network with Sentiment Fusion for Multimodal Sentiment Analysis

TeFNA: Text-centered Fusion Network with crossmodal Attention for multimodal sentiment analysis

SKEAFN: Sentiment Knowledge Enhanced Attention Fusion Network for multimodal sentiment analysis

MFSC: A Multimodal Aspect-Level Sentiment Classification Framework with Multi-Image Gate and Fusion Networks

Multimodal Sentiment Analysis Using Multi-tensor Fusion Network with Cross-modal Modeling

Multi-Grained Fusion Network with Self-Distillation for Aspect-Based Multimodal Sentiment Analysis

Cross-Modal Sentiment Analysis of Text and Video Based on Bi-GRU Cyclic Network and Correlation Enhancement

BAFN: Bi-Direction Attention Based Fusion Network for Multimodal Sentiment Analysis