Abstract:In this paper, we target the CCPR 2016 Multimodal Emotion Recognition Challenge (MEC 2016) which is based on the Chinese Natural Audio-Visual Emotion Database (CHEAVD) of movies and TV programs showing (nearly) spontaneous human emotions. Low level descriptors (LLDs) are proposed as audio features. As visual features, we propose using histogram of oriented gradients (HOG), local phase quantisation (LPQ), shape features and behavior-related features such as head pose and eye gaze. The visual features are post processed to delete or smooth the all-zero feature vector segments. Single modal emotion recognition is performed using fully connected hidden Markov models (HMMs). For multimodal emotion recognition, two schemes are proposed: in the first scheme the normalized probability vectors from the HMMs are input to a support vector machine (SVM) for final recognition. For the second scheme, the final emotion is estimated using audio or video features depending if the face has been detected on the full video. Moreover, to make full use of the labeled data and to overcome the problem of unbalanced data, we use the training set and validation set together to train the HMMs and SVMs with parameters optimized via cross-validation experiments. Experimental results on the test set show that the macro average precisions (MAPs) of audio, visual, and multimodal emotion recognition reach \(42.85\,\%\), \(54.24\,\%\), and \(53.90\,\%\), respectively, which are much higher than the corresponding baseline results of \(24.02\,\%\), \(34.28\,\%\), and \(30.63\,\%\).

MEC 2016: The Multimodal Emotion Recognition Challenge of CCPR 2016.

MEC 2017: Multimodal Emotion Recognition Challenge

Audio Visual Recognition of Spontaneous Emotions In-the-Wild.

Emotion Recognition in Videos via Fusing Multimodal Features.

Chinese Multimodal Emotion Recognition in Deep and Traditional Machine Leaming Approaches

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Emotion Recognition Using Multimodal Features

Audio-Visual Emotion Recognition with Capsule-like Feature Representation and Model-Based Reinforcement Learning

MEGC2024: ACM Multimedia 2024 Facial Micro-Expression Grand Challenge

MEGC2023: ACM Multimedia 2023 ME Grand Challenge

Audio-Video Based Multimodal Emotion Recognition Using Svms and Deep Learning

MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning.

Ensemble System for Multimodal Emotion Recognition Challenge (MEC 2017)

Audio And Face Video Emotion Recognition In The Wild Using Deep Neural Networks And Small Datasets

MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning

Investigation of Multimodal Features, Classifiers and Fusion Methods for Emotion Recognition

Multi-Modal Multi-Cultural Dimensional Continues Emotion Recognition In Dyadic Interactions

Multimodal Utterance-level Affect Analysis using Visual, Audio and Text Features

Multimodal Emotion Recognition in Multi-Cultural Conditions

Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations

Video Emotion Recognition in the Wild Based on Fusion of Multimodal Features