Abstract:In this paper, we study multimodal metaphor detection to obtain real semantic meaning from multiple heterogeneous information sources. The existing approaches mainly suffer from two drawbacks. (1) They focus on textual aspects, overlooking the characteristics of visual metaphor information. (2) Efficient methods for fusing multimodal metaphor features are lacking. To address the first issue, we propose a visual information enhancement method based on dual-granularity visual feature fusion , obtaining complete metaphorical visual features. To achieve bidirectional interaction among multimodal metaphor features, we further develop a multi-interactive crossmodal residual network (MCRN) that fuses the consistent and complementary information between different modalities and design a progressive fusion strategy to enhance the iterative fusion ability of the model. We extensively evaluate the proposed method on the popular Met-meme metaphor detection benchmark, outperforming the existing state-of-the-art methods by a large margins; i.e., we achieve F1 score improvements ranging from 1.47% to 2.55% under different languages. In addition, we further extend the evaluation to the Sarcasm dataset to validate the ability of the model to perceive semantic contrasts and meaning transformations, and the experimental results are superior to those of a strong baseline model.

ALBERT-BiLSTM for Sequential Metaphor Detection

BiLSTM-Based Models for Metaphor Detection.

Neural Metaphor Detecting with CNN-LSTM Model

Multi-task metaphor detection based on linguistic theory

Metaphor Detection via Linguistics Enhanced Siamese Network.

Thu ngn at naacl-2018 metaphor shared task: Neural metaphor detecting with cnn-lstm model

Sensorimotor Enhanced Neural Network for Metaphor Detection.

Enhancing Metaphor Detection through Soft Labels and Target Word Prediction

Metaphor Detection via Explicit Basic Meanings Modelling

Enhancing Metaphor Recognition of Literary Works in Applied Artificial Intelligence: A Multi-Level Approach with Bi-LSTM and CNN Fusion

Metaphor Detection using Deep Contextualized Word Embeddings

Metaphor identification: A contextual inconsistency based neural sequence labeling approach

Using Conceptual Norms for Metaphor Detection

Automatic Metaphor Recognition Based on Semantic Relation Patterns

A New Method Of Metaphor Recognition For A-Is-B Model In Chinese Sentences

A Match Made in Heaven: A Multi-task Framework for Hyperbole and Metaphor Detection

Chinese Metaphor Recognition Using a Multi-stage Prompting Large Language Model

VIEMF: Multimodal metaphor detection via visual information enhancement with multimodal fusion

Curriculum-style Data Augmentation for LLM-based Metaphor Detection

Unveiling the Invisible: Captioning Videos with Metaphors

Unsupervised Chinese Verb Metaphor Recognition Based On Selectional Preferences