Abstract:Emotion recognition in conversations (ERC) has emerged as an important research area in Natural Language Processing and Affective Computing, focusing on accurately identifying emotions within the conversational utterance. Conventional approaches typically rely on labeled training samples for fine-tuning pre-trained language models (PLMs) to enhance classification performance. However, the limited availability of labeled data in real-world scenarios poses a significant challenge, potentially resulting in diminished model performance. In response to this challenge, we present the Multi-modal Attentive Prompt (MAP) learning framework, tailored specifically for few-shot emotion recognition in conversations. The MAP framework consists of four integral modules: multi-modal feature extraction for the sequential embedding of text, visual, and acoustic inputs; a multi-modal prompt generation module that creates six manually-designed multi-modal prompts; an attention mechanism for prompt aggregation; and an emotion inference module for emotion prediction. To evaluate our proposed model’s efficacy, we conducted extensive experiments on two widely recognized benchmark datasets, MELD and IEMOCAP. Our results demonstrate that the MAP framework outperforms state-of-the-art ERC models, yielding notable improvements of 3.5% and 0.4% in micro F1 scores. These findings highlight the MAP learning framework’s ability to effectively address the challenge of limited labeled data in emotion recognition, offering a promising strategy for improving ERC model performance.

Emotion recognition in conversations with emotion shift detection based on multi-task learning

BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversation

InstructERC: Reforming Emotion Recognition in Conversation with Multi-task Retrieval-Augmented Large Language Models

InstructERC: Reforming Emotion Recognition in Conversation with a Retrieval Multi-task LLMs Framework

Enhancing Emotion Recognition in Conversation Via Multi-view Feature Alignment and Memorization

Multi-Modal Attentive Prompt Learning for Few-shot Emotion Recognition in Conversations

Enhancing Emotion Recognition in Conversation through Emotional Cross-Modal Fusion and Inter-class Contrastive Learning

Emotion Recognition in Conversation Using Capsule Networks and Gated Recurrent Units

SI-LSTM: Speaker Hybrid Long-short Term Memory and Cross Modal Attention for Emotion Recognition in Conversation

ERNetCL: A novel emotion recognition network in textual conversation based on curriculum learning strategy

CFN-ESA: A Cross-Modal Fusion Network with Emotion-Shift Awareness for Dialogue Emotion Recognition.

EmotionIC: emotional inertia and contagion-driven dependency modeling for emotion recognition in conversation

Multimodal Prompt Transformer with Hybrid Contrastive Learning for Emotion Recognition in Conversation

A Hierarchical Transformer with Speaker Modeling for Emotion Recognition in Conversation

A Contextual Attention Network for Multimodal Emotion Recognition in Conversation

Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations

Dialogue emotion model based on local–global context encoder and commonsense knowledge fusion attention

MultiEMO: an Attention-Based Correlation-Aware Multimodal Fusion Framework for Emotion Recognition in Conversations.

An Emotion Evolution Network for Emotion Recognition in Conversation

Real-Time Video Emotion Recognition Based on Reinforcement Learning and Domain Knowledge

HAAN-ERC: Hierarchical Adaptive Attention Network for Multimodal Emotion Recognition in Conversation