Abstract:Humanoid robots often struggle to express the intricate and authentic facial expressions characteristic of humans, potentially hampering user engagement. To address this challenge, we introduce a comprehensive two-stage methodology to empower our autonomous affective robot with the capacity to exhibit rich and natural facial expressions. In the initial stage, we present an innovative action unit (AU) driven facial expression disentangled synthesis method, enabling the generation of nuanced robot facial expression images guided by AUs. By harnessing facial AUs within a framework of weakly supervised learning, we effectively surmount the scarcity of paired training data (comprising source and target facial expression images). To preserve the integrity of AUs while mitigating identity interference, we leverage a latent facial attribute space to disentangle expression-related and expression-unrelated cues, employing solely the former for expression synthesis. In the subsequent phase, we actualize an affective robot endowed with multifaceted degrees of freedom for facial movements, facilitating the embodiment of the synthesized fine-grained facial expressions. We devise a specialized motor command mapping network that serves as a conduit between the generated expression images and the robot's realistic facial responses. By utilizing the physical motor positions as constraints, we refine the prediction of precise motor commands from the robot's generated facial expressions. This refinement process ensures that the robot's facial movements authentically express accurate and natural expressions. Finally, qualitative and quantitative evaluations on the benchmarking Emotionet dataset verify the effectiveness of the proposed generation method. Results on the self-developed affective robot indicate that our method achieves a promising generation of specific facial expressions with given AUs, significantly enhancing the affective human-robot interaction.

Facial Expression Imitation Method for Humanoid Robot Based on Smooth-Constraint Reversed Mechanical Model (SRMM)

Design and Realization of a Multi-DoF Robotic Head for Affective Humanoid Facial Expression Imitation.

Online Facial Expression Imitation for Humanoid Robot Based on RBF Neural Network

Human-like Facial Expression Imitation for Humanoid Robot Based on Recurrent Neural Network.

Motion Imitation of a Humanoid Robot Via Pose Estimation

Cgan Based Facial Expression Recognition for Human-Robot Interaction

A Two-Stage Facial Kinematic Control Strategy for Humanoid Robots Based on Keyframe Detection and Keypoint Cubic Spline Interpolation

A Facial Expression Emotion Recognition Based Human-robot Interaction System

Facial expression control method for humanoid expression robot based on multi-layer gate recurrent unit network

Human-robot facial coexpression

Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models

Real-Time Robotic Mirrored Behavior of Facial Expressions and Head Motions Based on Lightweight Networks

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Video Tracked Facial Expression Animation

Facial Expression Animation Based on Physical Model

EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning

Driving Animatronic Robot Facial Expression From Speech

Unlocking Human-Like Facial Expressions in Humanoid Robots: A Novel Approach for Action Unit Driven Facial Expression Disentangled Synthesis

Neuromuscular Control of the Face-Head-Neck Biomechanical Complex With Learning-Based Expression Transfer From Images and Videos

ExpressionBot: An Emotive Lifelike Robotic Face for Face-to-Face Communication

Automatic Generation of Robot Facial Expressions with Preferences.