Abstract:As an emerging research topic for proximity service (ProSe), automatic emotion recognition enables the machines to understand the emotional changes of human beings which can not only facilitate natural, effective, seamless, and advanced human–robot interaction or human–computer interface but also promote emotional health. Facial expression recognition (FER) is a vital task for emotion recognition. However, significant gap between human and machine exists in FER task. In this paper, we present a conditional generative adversarial network-based approach to alleviate the intra-class variations by individually controlling the facial expressions and learning the generative and discriminative representations simultaneously. The proposed framework consists of a generator G and three discriminators ( Di , Da , and Dexp ). The generator G transforms any query face image into another prototypic facial expression image with other factors preserved. Armed with action units condition, the generator G pays more attention to information relevant to facial expression. Three loss functions ( $L_{I}$ , La , and Lexp ) corresponding to the three discriminators ( Di , Da , and Dexp ) were designed to learn generative and discriminative representations. Moreover, after rendering the generated expression back to its original facial expression, cycle consistency loss is also applied to guarantee the identity and produce more constrained visual representations. Optimized by combining both synthesis and classification loss functions, the learnt representation is explicitly disentangled from other variations such as identity, head pose, and illumination. Qualitative and quantitative experimental results demonstrate the proposed FER system is effective for expression recognition.

An Ensemble Approach for Facial Expression Analysis in Video

Cgan Based Facial Expression Recognition for Human-Robot Interaction

An Effective Ensemble Learning Framework for Affective Behaviour Analysis

A Unified Approach to Facial Affect Analysis: the MAE-Face Visual Representation.

Transformer-based Multimodal Information Fusion for Facial Expression Analysis

Affective Behaviour Analysis via Integrating Multi-Modal Knowledge

Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers

Multi-modal Facial Affective Analysis based on Masked Autoencoder

Audio-visual Based Emotion Recognition-a New Approach

Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos

Affective Behaviour Analysis via Progressive Learning

Valence and Arousal Estimation Based on Multimodal Temporal-Aware Features for Videos in the Wild

Sensing emotional valence and arousal dynamics through automated facial action unit analysis

Valence-Arousal Model based Emotion Recognition using EEG, peripheral physiological signals and Facial Expression

A Multi-term and Multi-task Analyzing Framework for Affective Analysis in-the-wild

AI in Pursuit of Happiness, Finding Only Sadness: Multi-Modal Facial Emotion Recognition Challenge

A robust multimodal approach for emotion recognition

Learning Expression Features via Deep Residual Attention Networks for Facial Expression Recognition From Video Sequences

Automatic Recognition of Facial Displays of Unfelt Emotions

Computer Vision Estimation of Emotion Reaction Intensity in the Wild

2D/3D Expression Generation Using Advanced Learning Techniques and the Emotion Wheel