Abstract:As an emerging research topic for proximity service (ProSe), automatic emotion recognition enables the machines to understand the emotional changes of human beings which can not only facilitate natural, effective, seamless, and advanced human–robot interaction or human–computer interface but also promote emotional health. Facial expression recognition (FER) is a vital task for emotion recognition. However, significant gap between human and machine exists in FER task. In this paper, we present a conditional generative adversarial network-based approach to alleviate the intra-class variations by individually controlling the facial expressions and learning the generative and discriminative representations simultaneously. The proposed framework consists of a generator G and three discriminators ( Di , Da , and Dexp ). The generator G transforms any query face image into another prototypic facial expression image with other factors preserved. Armed with action units condition, the generator G pays more attention to information relevant to facial expression. Three loss functions ( $L_{I}$ , La , and Lexp ) corresponding to the three discriminators ( Di , Da , and Dexp ) were designed to learn generative and discriminative representations. Moreover, after rendering the generated expression back to its original facial expression, cycle consistency loss is also applied to guarantee the identity and produce more constrained visual representations. Optimized by combining both synthesis and classification loss functions, the learnt representation is explicitly disentangled from other variations such as identity, head pose, and illumination. Qualitative and quantitative experimental results demonstrate the proposed FER system is effective for expression recognition.

Facial Expression Editing with Continuous Emotion Labels

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Continuously Controllable Facial Expression Editing in Talking Face Videos

Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor

ExprGAN: Facial Expression Editing With Controllable Expression Intensity

Self-supervised Deformation Modeling for Facial Expression Editing

Semantic Facial Expression Editing using Autoencoded Flow

A Continuous Emotional Editing Model for Talking Head Videos Based on Decoupling Texture and Geometry

Toward Fine-grained Facial Expression Manipulation

A Unified and Interpretable Emotion Representation and Expression Generation

Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis

Leveraging the Deep Learning Paradigm for Continuous Affect Estimation from Facial Expressions

Neural Emotion Director: Speech-preserving semantic control of facial expressions in "in-the-wild" videos

LES-Talker: Fine-Grained Emotion Editing for Talking Head Generation in Linear Emotion Space

Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detection

Fine-Grained Facial Expression Analysis Using Dimensional Emotion Model

Enhancing facial expression recognition through generative adversarial networks-based augmentation

Joint Expression Synthesis and Representation Learning for Facial Expression Recognition

Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks

CAGE: Circumplex Affect Guided Expression Inference