Abstract:Automatic emotion recognition is a challenging task which can make great impact on improving natural human computer interactions. In this paper, we present our solutions for the Cross-cultural Emotion Sub-challenge (CES) of Audio/Visual Emotion Challenge (AVEC) 2018. The videos were recorded in dyadic human-human interaction scenarios. In these complicated scenarios, a person's emotion state will be influenced by the interlocutor's behaviors, such as talking style/prosody, speech content, facial expression and body language. In this paper, we highlight two aspects of our solutions: 1) we explore multiple modalities's efficient deep learning features and use the LSTM network to capture the long-term temporal information. 2) we propose several multimodal interaction strategies to imitate the real interaction patterns for exploring which modality information of the interlocutor is effective, and we find the best interaction strategy which can make full use of the interlocutor's information. Our solutions achieve the best CCC performance of 0.704 and 0.783 on arousal and valence respectively on the challenge testing set of German, which significantly outperform the baseline system with corresponding CCC of 0.524 and 0.577 on arousal and valence, and which outperform the winner of the AVEC2017 with corresponding CCC of 0.675 and 0.756 on arousal and valence. The experimental results show that our proposed interaction strategies have strong generalization ability and can bring more robust performance.

Enhancing Facial Expression Recognition through Dual-Direction Attention Mixed Feature Networks: Application to 7th ABAW Challenge

A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition

HSEmotion Team at the 7th ABAW Challenge: Multi-Task Learning and Compound Facial Expression Recognition

Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challenge

HSEmotion Team at the 6th ABAW Competition: Facial Expressions, Valence-Arousal and Emotion Intensity Prediction

Twin attention based multi-task convolutional bidirectional long short term memory for facial expression recognition

Emotic Masked Autoencoder with Attention Fusion for Facial Expression Recognition

Multi-Task Learning for Emotion Descriptors Estimation at the fourth ABAW Challenge

SUN Team's Contribution to ABAW 2024 Competition: Audio-visual Valence-Arousal Estimation and Expression Recognition

Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition

Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition

Adaptive Fusion Attention Network for Facial Expression Recognition

Transformer-based Multimodal Information Fusion for Facial Expression Analysis

A Dual Attention-based Modality-Collaborative Fusion Network for Emotion Recognition

Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network

A facial expression recognition network based on attention double branch enhanced fusion

Continuous-Time Audiovisual Fusion with Recurrence vs. Attention for In-The-Wild Affect Recognition

3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network

Multi-Modal Multi-Cultural Dimensional Continues Emotion Recognition In Dyadic Interactions

Facial Expression Recognition Through Cross-Modality Attention Fusion

Affective Behavior Analysis using Task-adaptive and AU-assisted Graph Network