Abstract:Aspect-level multimodal sentiment analysis aims to ascertain the sentiment polarity of a given aspect from a text review and its accompanying image. Despite substantial progress made by existing research, aspect-level multimodal sentiment analysis still faces several challenges: (1) Inconsistency in feature granularity between the text and image modalities poses difficulties in capturing corresponding visual representations of aspect words. This inconsistency may introduce irrelevant or redundant information, thereby causing noise and interference in sentiment analysis. (2) Traditional aspect-level sentiment analysis predominantly relies on the fusion of semantic and syntactic information to determine the sentiment polarity of a given aspect. However, introducing image modality necessitates addressing the semantic gap in jointly understanding sentiment features in different modalities. To address these challenges, a multi-granularity visual-textual feature fusion model (MG-VTFM) is proposed to enable deep sentiment interactions among semantic, syntactic, and image information. First, the model introduces a multi-granularity hierarchical graph attention network that controls the granularity of semantic units interacting with images through constituent tree. This network extracts image sentiment information relevant to the specific granularity, reduces noise from images and ensures sentiment relevance in single-granularity cross-modal interactions. Building upon this, a multilayered graph attention module is employed to accomplish multi-granularity sentiment fusion, ranging from fine to coarse. Furthermore, a progressive multimodal attention fusion mechanism is introduced to maximize the extraction of abstract sentiment information from images. Lastly, a mapping mechanism is proposed to align cross-modal information based on aspect words, unifying semantic spaces across different modalities. Our model demonstrates excellent overall performance on two datasets.

Modeling Inter-aspect Relationship with Conjunction for Aspect-Based Sentiment Analysis

Modeling Inter-Aspect Dependencies with a Non-temporal Mechanism for Aspect-Based Sentiment Analysis.

A span-based model for aspect terms extraction and aspect sentiment classification

Effective inter-aspect words modeling for aspect-based sentiment analysis

Modeling Inter-Aspect Relations with Clause and Contrastive Learning for Aspect-Based Sentiment Analysis

Modeling Adaptive Inter-Task Feature Interactions Via Sentiment-Aware Contrastive Learning for Joint Aspect-Sentiment Prediction

Fusing Sentiment Knowledge and Inter-Aspect Dependency Based on Gated Mechanism for Aspect-Level Sentiment Classification

Aspect-level Sentiment Classification Based on Aspect-Oriented Information and Inter-Aspect Relations

Sequence Prediction Model for Aspect-Level Sentiment Classification.

Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal Relation Detection.

A Joint Model for Sentiment Classification and Opinion Words Extraction

Multifeature Interactive Fusion Model for Aspect-Based Sentiment Analysis

Sentiment-Aspect Analysis Through Semi-Supervised Topic Modeling

Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model

Research on Aspect-Level Sentiment Analysis Based on Adversarial Training and Dependency Parsing

A Unified Labeling Model for Open-Domain Aspect-Based Sentiment Analysis

Multiple-element joint detection for Aspect-Based Sentiment Analysis

Bidirectional Complementary Correlation-Based Multimodal Aspect-Level Sentiment Analysis

Multi-granularity Visual-Textual Jointly Modeling for Aspect-Level Multimodal Sentiment Analysis

Modeling Category Semantic and Sentiment Knowledge for Aspect-Level Sentiment Analysis

Aspect-Pair Supervised Contrastive Learning for aspect-based sentiment analysis