Abstract:Humanoid social robots have an increasingly prominent place in today’s world. Their acceptance in social and emotional human–robot interaction (HRI) scenarios depends on their ability to convey well recognized and believable emotional expressions to their human users. In this article, we incorporate recent findings from psychology, neuroscience, human–computer interaction, and HRI, to examine how people recognize and respond to emotions displayed by the body and voice of humanoid robots, with a particular emphasis on the effects of incongruence. In a social HRI laboratory experiment, we investigated contextual incongruence (i.e., the conflict situation where a robot’s reaction is incongrous with the socio-emotional context of the interaction) and cross-modal incongruence (i.e., the conflict situation where an observer receives incongruous emotional information across the auditory (vocal prosody) and visual (whole-body expressions) modalities). Results showed that both contextual incongruence and cross-modal incongruence confused observers and decreased the likelihood that they accurately recognized the emotional expressions of the robot. This, in turn, gives the impression that the robot is unintelligent or unable to express “empathic” behaviour and leads to profoundly harmful effects on likability and believability. Our findings reinforce the need of proper design of emotional expressions for robots that use several channels to communicate their emotional states in a clear and effective way. We offer recommendations regarding design choices and discuss future research areas in the direction of multimodal HRI.

Analysis of Influencing Factors on Humanoid Robots' Emotion Expressions by Body Language.

Research on Emotion Body Language Model of the Humanoid Robot

The Effect of a Humanoid Robot’s Emotional Behaviors on Users’ Emotional Responses: Evidence from Pupillometry and Electroencephalography Measures

Multi-Modal Based Fuzzy Atmosfield in Human-Robot Interaction

Vocal emotion of humanoid robots: a study from brain mechanism.

Improving Human-Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

Evaluation of Robot Emotion Expressions for Human–Robot Interaction

Enhancing the Mobile Humanoid Robot's Emotional Expression with Affective Vertical-Oscillations

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

Emotional States Based 3-D Fuzzy Atmosfield for Casual Communication Between Humans and Robots

A Multimodal Emotional Communication Based Humans-Robots Interaction System

Dynamic Emotion Understanding in Human–Robot Interaction Based on Two-Layer Fuzzy SVR-TS Model

Robots’ “Woohoo” and “Argh” can Enhance Users’ Emotional and Social Perceptions: An Exploratory Study on Non-Lexical Vocalizations and Non-Linguistic Sounds

Can my robotic home cleaner be happy? Issues about emotional expression in non-bio-inspired robots

Emotion Expression of Robot with Personality

Estimating Emotional Intensity from Body Poses for Human-Robot Interaction

A Facial Expression Emotion Recognition Based Human-robot Interaction System

Data-driven emotional body language generation for social robotics

Communication Atmosphere in Humans and Robots Interaction Based on Fuzzy Analytical Hierarchy Process

Perception of Emotions in Human and Robot Faces: Is the Eye Region Enough?

Multimodal Integration of Emotional Signals from Voice, Body, and Context: Effects of (In)Congruence on Emotion Recognition and Attitudes Towards Robots