Abstract:Social media comments are no longer in a single textual modality, but heterogeneous data in multiple modalities, such as vision, sound, and text, which is why multimodal sentiment analysis strategies has been introduced. However, among the multimodal sentiment analysis domains, a majority of the current multimodal sentiment analysis models employ the Transformer architecture due to its great impact and benefits, thereby leading to an augmentation in resource overhead. In this paper, a multimodal attention fusion (MAF) network model is proposed for sentiment analysis of multimodal data. MAF is mainly composed of the cross attention and residual unit. The Cross Attention Unit is designed to select one core modality out of three modes, while the remaining modes serve as base modalities. The core modality is then combined with the base modality information to facilitate significant interaction between the two modalities, resulting in three sets of two-by-two attention computations. Moreover, a residual unit is employed to integrate the overall information into the attentional information. This approach not only enables modality-to-modality interaction, but also supplements the overall information. In the end, experiments are conducted on two publicly available multimodal sentiment analysis datasets from Carnegie Mellon University(CMU), CMU-MOSEI (abbreviated as MOSEI) and CMU-MOSI (abbreviated as MOSI), to validate that the method achieves high performance while removing the complex structure, and is comparable to the State-Of-The-Art(SOTA) model with high-performance A100 and V100 Graphics Processing Units(GPU) in an ordinary hardware environment.

Multi-layer cross-modality attention fusion network for multimodal sentiment analysis

Sentiment Analysis of Social Media Comments Based on Multimodal Attention Fusion Network

A Multimodal Sentiment Analysis Method Integrating Multi-Layer Attention Interaction and Multi-Feature Enhancement

Exploring Multimodal Sentiment Analysis via CBAM Attention and Double-layer BiLSTM Architecture

Multi-Modal Sentiment Analysis Based on Image and Text Fusion Based on Cross-Attention Mechanism

Multi-Feature Fusion Multi-Modal Sentiment Analysis Model Based on Cross-Attention Mechanism

Multimodal sentiment analysis based on multiple attention

A text guided multi-task learning network for multimodal sentiment analysis

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism

Multimodal Sentiment Analysis of Social Media Based on Top-Layer Fusion

Multimodal Sentiment Analysis Using Multi-tensor Fusion Network with Cross-modal Modeling

Multi-Channel Attentive Graph Convolutional Network with Sentiment Fusion for Multimodal Sentiment Analysis

Multimodal Transformer with Adaptive Modality Weighting for Multimodal Sentiment Analysis

A cross modal hierarchical fusion multimodal sentiment analysis method based on multi-task learning

TETFN: A Text Enhanced Transformer Fusion Network for Multimodal Sentiment Analysis

Multimodal Sentiment Analysis Based on Attentional Temporal Convolutional Network and Multi-Layer Feature Fusion

Multimodal Sentiment Analysis Based on Pre-LN Transformer Interaction

Multi-level Multi-task Representation Learning with Adaptive Fusion for Multimodal Sentiment Analysis

Multimodal Mutual Attention-Based Sentiment Analysis Framework Adapted to Complicated Contexts

Multi-level Attention Map Network for Multimodal Sentiment Analysis

Text-oriented Modality Reinforcement Network for Multimodal Sentiment Analysis from Unaligned Multimodal Sequences