Abstract:A plentiful number of facial expressions is the basis of natural human–robot interaction for high-fidelity humanoid robots. The facial expression imitation of humanoid robots involves the transmission of human facial expression data to servos situated within the robot's head. These data drive the servos to manipulate the skin, thereby enabling the robot to exhibit various facial expressions. However, since the mechanical transmission rate cannot keep up with the data processing rate, humanoid robots often suffer from jitters in the imitation. We conducted a thorough analysis of the transmitted facial expression sequence data and discovered that they are extremely redundant. Therefore, we designed a two-stage strategy for humanoid robots based on facial keyframe detection and facial keypoint detection to achieve more natural and smooth expression imitation. We first built a facial keyframe detection model based on ResNet-50, combined with optical flow estimation, which can identify key expression frames in the sequence. Then, a facial keypoint detection model is used on the keyframes to obtain the facial keypoint coordinates. Based on the coordinates, the cubic spline interpolation method is used to obtain the motion trajectory parameters of the servos, thus realizing the robust control of the humanoid robot's facial expression. Experiments show that, unlike before where the robot's imitation would stutter at frame rates above 25 fps, our strategy allows the robot to maintain good facial expression imitation similarity (cosine similarity of 0.7226), even at higher frame rates.

Research on humanoid robot expressions based on mood control

Design and Realization of a Multi-DoF Robotic Head for Affective Humanoid Facial Expression Imitation.

Contactless Interaction System Based on Facial Expression Recognition for Humanoid Piano Robot

Mechanical Design of a Humanoid Robot Face

A Facial Expression Emotion Recognition Based Human-robot Interaction System

A Multimodal Emotional Communication Based Humans-Robots Interaction System

Human-robot facial coexpression

A Two-Stage Facial Kinematic Control Strategy for Humanoid Robots Based on Keyframe Detection and Keypoint Cubic Spline Interpolation

Multi-Modal Based Fuzzy Atmosfield in Human-Robot Interaction

Emotion Expression of Robot with Personality

Emotional Communication Robot Based on 3D Face Model and ASR Technology

Research on Emotion Body Language Model of the Humanoid Robot

Creating the optimal design approach of facial expression for the elderly intelligent service robot

ExpressionBot: An Emotive Lifelike Robotic Face for Face-to-Face Communication

Enhancing the Mobile Humanoid Robot's Emotional Expression with Affective Vertical-Oscillations

Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models

Emotional States Based 3-D Fuzzy Atmosfield for Casual Communication Between Humans and Robots

Research on the embedded system of facial expression recognition based on HMM

Design and Control of 26 Freedom Intelligent Humanoid Robot

EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning

Multi-Modal Hierarchical Empathetic Framework for Social Robots With Affective Body Control