Abstract:Social media comments are no longer in a single textual modality, but heterogeneous data in multiple modalities, such as vision, sound, and text, which is why multimodal sentiment analysis strategies has been introduced. However, among the multimodal sentiment analysis domains, a majority of the current multimodal sentiment analysis models employ the Transformer architecture due to its great impact and benefits, thereby leading to an augmentation in resource overhead. In this paper, a multimodal attention fusion (MAF) network model is proposed for sentiment analysis of multimodal data. MAF is mainly composed of the cross attention and residual unit. The Cross Attention Unit is designed to select one core modality out of three modes, while the remaining modes serve as base modalities. The core modality is then combined with the base modality information to facilitate significant interaction between the two modalities, resulting in three sets of two-by-two attention computations. Moreover, a residual unit is employed to integrate the overall information into the attentional information. This approach not only enables modality-to-modality interaction, but also supplements the overall information. In the end, experiments are conducted on two publicly available multimodal sentiment analysis datasets from Carnegie Mellon University(CMU), CMU-MOSEI (abbreviated as MOSEI) and CMU-MOSI (abbreviated as MOSI), to validate that the method achieves high performance while removing the complex structure, and is comparable to the State-Of-The-Art(SOTA) model with high-performance A100 and V100 Graphics Processing Units(GPU) in an ordinary hardware environment.

A text guided multi-task learning network for multimodal sentiment analysis

Multimodal Sentiment Analysis With Two-Phase Multi-Task Learning

Text-oriented Modality Reinforcement Network for Multimodal Sentiment Analysis from Unaligned Multimodal Sequences

Low-rank tensor fusion and self-supervised multi-task multimodal sentiment analysis

Multi-layer cross-modality attention fusion network for multimodal sentiment analysis

A Multimodal Sentiment Analysis Method Integrating Multi-Layer Attention Interaction and Multi-Feature Enhancement

A cross modal hierarchical fusion multimodal sentiment analysis method based on multi-task learning

M$^{3}$SA: Multimodal Sentiment Analysis Based on Multi-Scale Feature Extraction and Multi-Task Learning

TETFN: A Text Enhanced Transformer Fusion Network for Multimodal Sentiment Analysis

TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment Analysis

Dynamic Weighted Multitask Learning and Contrastive Learning for Multimodal Sentiment Analysis

Semantic-specific multimodal relation learning for sentiment analysis

Cross-Modal Multitask Transformer for End-to-End Multimodal Aspect-Based Sentiment Analysis

Multimodal sentiment analysis based on multiple attention

Sentiment Analysis of Social Media Comments Based on Multimodal Attention Fusion Network

Deep Modular Co-Attention Shifting Network for Multimodal Sentiment Analysis.

Text-Centric Multimodal Contrastive Learning for Sentiment Analysis

A Fine-Grained Modal Label-Based Multi-Stage Network for Multimodal Sentiment Analysis.

Learning Modality-Complementary and Eliminating-Redundancy Representations with Multi-Task Learning for Multimodal Sentiment Analysis

Multi-level Multi-task Representation Learning with Adaptive Fusion for Multimodal Sentiment Analysis