Abstract:This paper presents a new approach for evaluating and controlling expressive humanoid robotic faces using open-source computer vision and machine learning methods. Existing research in Human-Robot Interaction lacks flexible and simple tools that are scalable for evaluating and controlling various robotic faces; thus, our goal is to demonstrate the use of readily available AI-based solutions to support the process. We use a newly developed humanoid robot prototype intended for medical training applications as a case example. The approach automatically captures the robot’s facial action units through a webcam during random motion, which are components traditionally used to describe facial muscle movements in humans. Instead of manipulating the actuators individually or training the robot to express specific emotions, we propose using action units as a means for controlling the robotic face, which enables a multitude of ways to generate dynamic motion, expressions, and behavior. The range of action units achieved by the robot is thus analyzed to discover its expressive capabilities and limitations and to develop a control model by correlating action units to actuation parameters. Because the approach is not dependent on specific facial attributes or actuation capabilities, it can be used for different designs and continuously inform the development process. In healthcare training applications, our goal is to establish a prerequisite of expressive capabilities of humanoid robots bounded by industrial and medical design constraints. Furthermore, to mediate human interpretation and thus enable decision-making based on observed cognitive, emotional, and expressive cues, our approach aims to find the minimum viable expressive capabilities of the robot without having to optimize for realism. The results from our case example demonstrate the flexibility and efficiency of the presented AI-based solutions to support the development of humanoid facial robots.

Design and Implementation of a Facial Character Analysis Algorithm for Humanoid Robots

Design and Realization of a Multi-DoF Robotic Head for Affective Humanoid Facial Expression Imitation.

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Facing the FACS—Using AI to Evaluate and Control Facial Action Units in Humanoid Robot Face Development

Intelligent Facial Emotion Recognition and Semantic-Based Topic Detection for A Humanoid Robot

Improving Human-Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

A Facial Expression Emotion Recognition Based Human-robot Interaction System

On effective human robot interaction based on recognition and association

Adaptive Facial Point Detection and Emotion Recognition for a Humanoid Robot

Facial Expression Recognition Based on Zero-Addition Pretext Training and Feature Conjunction-Selection Network in Human–Robot Interaction

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Facial Expressions Recognition for Human-Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer

Hierarchical Co-Consistency Quantization and Information Refining Binary Network for Facial Expression Recognition in Human–Robot Interaction

Emotional States Based 3-D Fuzzy Atmosfield for Casual Communication Between Humans and Robots

Emotion-Age-Gender-Nationality Based Intention Understanding in Human–Robot Interaction Using Two-Layer Fuzzy Support Vector Regression

Intelligent Appearance and shape based facial emotion recognition for a humanoid robot

An Automated Research for Emotion Recognition and Generation

ExpressionBot: An Emotive Lifelike Robotic Face for Face-to-Face Communication

NAO Robot Learns to Interact with Humans through Imitation Learning from Video Observation

A Comprehensive Multimodal Humanoid System for Personality Assessment Based on the Big Five Model

Research and Implementation of Human-Computer Interaction System Based on Human Body Attitude Recognition Algorithm