Abstract:Multimodal Aspect-Based Sentiment Analysis (MABSA) technology aims to utilize both textual and visual modalities to achieve Multimodal Aspect Term Extraction (MATE) and Multimodal Aspect Sentiment Classification (MASC) in tweets. Current research has overlooked the impact of noise from irrelevant regions in images on model performance. Additionally, there has been insufficient utilization of the textual information contained within images and the syntactic features of sentences. In this paper, we propose a Target-oriented Cross Modal Transformer (TCMT) for MABSA. The model consists of a textual auxiliary module, a visual auxiliary module, and a main module: the textual aspect-sentiment extraction module, the visual aspect-sentiment prediction module, and the textual-visual alignment cross-modal module. In the textual auxiliary module, we utilize syntactic features to assist the model in identifying the boundaries of multi-word aspect terms and employ Optical Character Recognition (OCR) technology to capture textual information contained within images. In the visual auxiliary module, we employ Adjective-Noun Pairs (ANPs) detection for supervised training of images. Additionally, we have improved the cross-modal Transformer structure by designing a GCN-based Transformer in the textual auxiliary module to learn syntactic graphs, and a CNN-based Transformer in the visual auxiliary module to focus more on important information in images. In the cross-modal MABSA module, we design a target-oriented interaction component to facilitate modal interaction learning and mitigate the impact of image noise, along with an alignment auxiliary component to optimize modal alignment training. We conducted extensive experiments on two publicly available benchmark datasets. The results demonstrate that the performance of the TCMT model is significantly superior to that of the baseline model, achieving state-of-the-art results. Both the textual auxiliary module and the visual auxiliary module effectively assist the cross-modal MABSA module in completing the task more efficiently.

Classifier-Based Tense Model for SMT

Adoption of Wireless Network and Artificial Intelligence Algorithm in Chinese-English Tense Translation

One Tense Per Scene: Predicting Tense in Chinese Conversations

Classifier-Based Text Simplification for Improved Machine Translation

Processing of Tense and Aspect in Chinese-English Machine Translation

Advancing Time Series Classification with Multimodal Language Modeling

Study on Machine Translation Teaching Model Based on Translation Parallel Corpus and Exploitation for Multimedia Asian Information Processing

Learning Bilingual Embedding Model for Cross-Language Sentiment Classification

Head-modifier relation based non-lexical reordering model for phrase-based translation

Using an Integrated Feature Set to Generalize and Justify the Chinese-to-English Transferring Rule of the ‘ZHE’ Aspect

TCMT: Target-oriented Cross Modal Transformer for Multimodal Aspect-Based Sentiment Analysis

Machine Translation for Machines: the Sentiment Classification Use Case

A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation.

Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition

Neural Machine Translation Advised by Statistical Machine Translation

Categorizing Semantic Representations for Neural Machine Translation

Detecting Syntactic Features of Translated Chinese

Adapting BERT for Target-Oriented Multimodal Sentiment Classification

Attention-based BILSTM network with part-of-speech features for Chinese text classification

Attention Mechanism and Context Modeling System for Text Mining Machine Translation

Language Model is a Branch Predictor for Simultaneous Machine Translation