Abstract:Multimodal Aspect-Based Sentiment Analysis (MABSA) technology aims to utilize both textual and visual modalities to achieve Multimodal Aspect Term Extraction (MATE) and Multimodal Aspect Sentiment Classification (MASC) in tweets. Current research has overlooked the impact of noise from irrelevant regions in images on model performance. Additionally, there has been insufficient utilization of the textual information contained within images and the syntactic features of sentences. In this paper, we propose a Target-oriented Cross Modal Transformer (TCMT) for MABSA. The model consists of a textual auxiliary module, a visual auxiliary module, and a main module: the textual aspect-sentiment extraction module, the visual aspect-sentiment prediction module, and the textual-visual alignment cross-modal module. In the textual auxiliary module, we utilize syntactic features to assist the model in identifying the boundaries of multi-word aspect terms and employ Optical Character Recognition (OCR) technology to capture textual information contained within images. In the visual auxiliary module, we employ Adjective-Noun Pairs (ANPs) detection for supervised training of images. Additionally, we have improved the cross-modal Transformer structure by designing a GCN-based Transformer in the textual auxiliary module to learn syntactic graphs, and a CNN-based Transformer in the visual auxiliary module to focus more on important information in images. In the cross-modal MABSA module, we design a target-oriented interaction component to facilitate modal interaction learning and mitigate the impact of image noise, along with an alignment auxiliary component to optimize modal alignment training. We conducted extensive experiments on two publicly available benchmark datasets. The results demonstrate that the performance of the TCMT model is significantly superior to that of the baseline model, achieving state-of-the-art results. Both the textual auxiliary module and the visual auxiliary module effectively assist the cross-modal MABSA module in completing the task more efficiently.

Incorporating Morphological Compostions with Transformer to Improve BERT

Incorporating Latent Meanings of Morphological Compositions to Enhance Word Embeddings

Expanding Expressivity in Transformer Models with MöbiusAttention

Segatron: Segment-Aware Transformer for Language Modeling and Understanding

MorphTE: Injecting Morphology in Tensorized Embeddings

Adapting BERT for Target-Oriented Multimodal Sentiment Classification

Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speech.

Enhancing Transformer with Sememe Knowledge.

Comateformer: Combined Attention Transformer for Semantic Sentence Matching

Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5

Enhanced Pre-Trained Transformer with Aligned Attention Map for Text Matching

CM-BERT

SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics

Better Character Language Modeling Through Morphology

CAT-BERT: A Context-Aware Transferable BERT Model for Multi-turn Machine Reading Comprehension.

TCMT: Target-oriented Cross Modal Transformer for Multimodal Aspect-Based Sentiment Analysis

Categorizing Semantic Representations for Neural Machine Translation

TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding

Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models

Incorporating emoji sentiment information into a pre-trained language model for Chinese and English sentiment analysis

Model Composition for Multimodal Large Language Models