Abstract:Humanoid robots often struggle to express the intricate and authentic facial expressions characteristic of humans, potentially hampering user engagement. To address this challenge, we introduce a comprehensive two-stage methodology to empower our autonomous affective robot with the capacity to exhibit rich and natural facial expressions. In the initial stage, we present an innovative action unit (AU) driven facial expression disentangled synthesis method, enabling the generation of nuanced robot facial expression images guided by AUs. By harnessing facial AUs within a framework of weakly supervised learning, we effectively surmount the scarcity of paired training data (comprising source and target facial expression images). To preserve the integrity of AUs while mitigating identity interference, we leverage a latent facial attribute space to disentangle expression-related and expression-unrelated cues, employing solely the former for expression synthesis. In the subsequent phase, we actualize an affective robot endowed with multifaceted degrees of freedom for facial movements, facilitating the embodiment of the synthesized fine-grained facial expressions. We devise a specialized motor command mapping network that serves as a conduit between the generated expression images and the robot's realistic facial responses. By utilizing the physical motor positions as constraints, we refine the prediction of precise motor commands from the robot's generated facial expressions. This refinement process ensures that the robot's facial movements authentically express accurate and natural expressions. Finally, qualitative and quantitative evaluations on the benchmarking Emotionet dataset verify the effectiveness of the proposed generation method. Results on the self-developed affective robot indicate that our method achieves a promising generation of specific facial expressions with given AUs, significantly enhancing the affective human-robot interaction.

What Do I Look Like? A Conditional GAN Based Robot Facial Self-Awareness Approach

Expression Conditional Gan for Facial Expression-to-Expression Translation.

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

Weight-Adapted Convolution Neural Network for Facial Expression Recognition

Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models

Comp-GAN

Data-driven emotional body language generation for social robotics

Self-Supervised GAN for Occluded Facial Expression Recognition

A Facial Expression Synthesis Method Based on Generative Adversarial Network.

Data Augmentation Using Conditional GANs for Facial Emotion Recognition

DEGAN: De-Expression Generative Adversarial Network for Expression-Invariant Face Recognition by Robot Vision

Facial Expression Transfer Based on Conditional Generative Adversarial Networks.

Automatic Generation of Robot Facial Expressions with Preferences.

Unlocking Human-Like Facial Expressions in Humanoid Robots: A Novel Approach for Action Unit Driven Facial Expression Disentangled Synthesis

XAGen: 3D Expressive Human Avatars Generation

GANalyzer: Analysis and Manipulation of GANs Latent Space for Controllable Face Synthesis

Facial Expression Recognition Based on Contextual Generative Adversarial Network