Abstract:The task of multi-modal emotion recognition in conversation (MERC) aims to analyze the genuine emotional state of each utterance based on the multi-modal information in the conversation, which is crucial for conversation understanding. Existing methods focus on using graph neural networks (GNN) to model conversational relationships and capture contextual latent semantic relationships. However, due to the complexity of GNN, existing methods cannot efficiently capture the potential dependencies between long-distance utterances, which limits the performance of MERC. In this paper, we propose an Efficient Long-distance Latent Relation-aware Graph Neural Network (ELR-GNN) for multi-modal emotion recognition in conversations. Specifically, we first use pre-extracted text, video and audio features as input to Bi-LSTM to capture contextual semantic information and obtain low-level utterance features. Then, we use low-level utterance features to construct a conversational emotion interaction graph. To efficiently capture the potential dependencies between long-distance utterances, we use the dilated generalized forward push algorithm to precompute the emotional propagation between global utterances and design an emotional relation-aware operator to capture the potential semantic associations between different utterances. Furthermore, we combine early fusion and adaptive late fusion mechanisms to fuse latent dependency information between speaker relationship information and context. Finally, we obtain high-level discourse features and feed them into MLP for emotion prediction. Extensive experimental results show that ELR-GNN achieves state-of-the-art performance on the benchmark datasets IEMOCAP and MELD, with running times reduced by 52\% and 35\%, respectively.

Effective Context Modeling Framework for Emotion Recognition in Conversations

Multivariate, Multi-Frequency and Multimodal: Rethinking Graph Neural Networks for Emotion Recognition in Conversation

GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition

Context- and Knowledge-Aware Graph Convolutional Network for Multimodal Emotion Recognition

CONSK-GCN - Conversational Semantic- and Knowledge-Oriented Graph Convolutional Network for Multimodal Emotion Recognition.

MMGCN: Multimodal Fusion Via Deep Graph Convolution Network for Emotion Recognition in Conversation

A Contextual Attention Network for Multimodal Emotion Recognition in Conversation

Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction

A Contextualized Real-Time Multimodal Emotion Recognition for Conversational Agents using Graph Convolutional Networks in Reinforcement Learning

Emotion Recognition in Conversation Based on a Dynamic Complementary Graph Convolutional Network

A Cross-Modality Context Fusion and Semantic Refinement Network for Emotion Recognition in Conversation.

Efficient Long-distance Latent Relation-aware Graph Neural Network for Multi-modal Emotion Recognition in Conversations

MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversations

LR-GCN: Latent Relation-Aware Graph Convolutional Network for Conversational Emotion Recognition

Multi-Scale Receptive Field Graph Model for Emotion Recognition in Conversations

Topics Guided Multimodal Fusion Network for Conversational Emotion Recognition

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

Knowledge-Aware Graph Convolutional Network with Utterance-Specific Window Search for Emotion Recognition in Conversations

Directed Acyclic Graph Network for Conversational Emotion Recognition

Multimodal Graph Learning with Framelet-Based Stochastic Configuration Networks for Emotion Recognition in Conversation