Abstract:Equipping humanoid robots with the capability to understand emotional states of human interactants and express emotions appropriately according to situations is essential for affective human-robot interaction. However, enabling current vision-aware multimodal emotion recognition models for affective human-robot interaction in the real-world raises embodiment challenges: addressing the environmental noise issue and meeting real-time requirements. First, in multiparty conversation scenarios, the noises inherited in the visual observation of the robot, which may come from either 1) distracting objects in the scene or 2) inactive speakers appearing in the field of view of the robot, hinder the models from extracting emotional cues from vision inputs. Secondly, realtime response, a desired feature for an interactive system, is also challenging to achieve. To tackle both challenges, we introduce an affective human-robot interaction system called UGotMe designed specifically for multiparty conversations. Two denoising strategies are proposed and incorporated into the system to solve the first issue. Specifically, to filter out distracting objects in the scene, we propose extracting face images of the speakers from the raw images and introduce a customized active face extraction strategy to rule out inactive speakers. As for the second issue, we employ efficient data transmission from the robot to the local server to improve realtime response capability. We deploy UGotMe on a human robot named Ameca to validate its real-time inference capabilities in practical scenarios. Videos demonstrating real-world deployment are available at <a class="link-external link-https" href="https://pi3-141592653.github.io/UGotMe/" rel="external noopener nofollow">this https URL</a>.

Evaluating Transferable Emotion Expressions for Zoomorphic Social Robots using VR Prototyping

Prototyping a Zoomorphic Interactive Robot Companion with Emotion Recognition and Affective Voice Interaction for Elderly People

Using Socially Expressive Mixed Reality Arms for Enhancing Low-Expressivity Robots

ExpressionBot: An Emotive Lifelike Robotic Face for Face-to-Face Communication

Enhancing the Mobile Humanoid Robot's Emotional Expression with Affective Vertical-Oscillations

A MultiModal Social Robot Toward Personalized Emotion Interaction

A Multimodal Emotional Communication Based Humans-Robots Interaction System

Design and Realization of a Multi-DoF Robotic Head for Affective Humanoid Facial Expression Imitation.

HexTouch: Affective Robot Touch for Complementary Interactions to Companion Agents in Virtual Reality

Recognizing Emotional Body Language Displayed by a Human-like Social Robot

Usability of a Robot's Realistic Facial Expressions and Peripherals in Autistic Children's Therapy

User Emotion and Perception of Virtual and Robot Companion Concepts for Shared Autonomous Mobility in Singapore

Evaluation of Robot Emotion Expressions for Human–Robot Interaction

Investigation of Relationships Between Embodiment Perceptions and Perceived Social Presence in Human–Robot Interactions

Crafting for Emotion Appropriateness in Affective Robotics: Examining the Practicality of the OCC Model

Real-time Emotion Appraisal with Circumplex Model for Human-Robot Interaction

UGotMe: An Embodied System for Affective Human-Robot Interaction

Using Affect as a Communication Modality to Improve Human-Robot Communication in Robot-Assisted Search and Rescue Scenarios

Multimodal Integration of Emotional Signals from Voice, Body, and Context: Effects of (In)Congruence on Emotion Recognition and Attitudes Towards Robots

Design and Evaluation of a Mobile Robotic Assistant for Emotional Learning in Individuals with ASD: Expert Evaluation Stage

Robots’ “Woohoo” and “Argh” can Enhance Users’ Emotional and Social Perceptions: An Exploratory Study on Non-Lexical Vocalizations and Non-Linguistic Sounds