Abstract:Humans are arguably innately prepared to comprehend others' emotional expressions from subtle body movements. If robots or computers can be empowered with this capability, a number of robotic applications become possible. Automatically recognizing human bodily expression in unconstrained situations, however, is daunting given the incomplete understanding of the relationship between emotional expressions and body movements. The current research, as a multidisciplinary effort among computer and information sciences, psychology, and statistics, proposes a scalable and reliable crowdsourcing approach for collecting in-the-wild perceived emotion data for computers to learn to recognize body languages of humans. To accomplish this task, a large and growing annotated dataset with 9,876 video clips of body movements and 13,239 human characters, named BoLD (Body Language Dataset), has been created. Comprehensive statistical analysis of the dataset revealed many interesting insights. A system to model the emotional expressions based on bodily movements, named ARBEE (Automated Recognition of Bodily Expression of Emotion), has also been developed and evaluated. Our analysis shows the effectiveness of Laban Movement Analysis (LMA) features in characterizing arousal, and our experiments using LMA features further demonstrate computability of bodily expression. We report and compare results of several other baseline methods which were developed for action recognition based on two different modalities, body skeleton, and raw image. The dataset and findings presented in this work will likely serve as a launchpad for future discoveries in body language understanding that will enable future robots to interact and collaborate more effectively with humans.

A Body Emotion-Based Human-Robot Interaction

A Sociable Human-robot Interaction Scheme Based on Body Emotion Analysis

The Design and Implementation of Natural Human-Robot Interaction System Based on Kinect Sensor

Beyond End-Effector: Utilizing High-Resolution Tactile Signals for Physical Human–Robot Interaction

Design and Realization of a Multi-DoF Robotic Head for Affective Humanoid Facial Expression Imitation.

Multi-Modal Based Fuzzy Atmosfield in Human-Robot Interaction

Multi-Robot Behavior Adaptation to Humans' Intention in Human-Robot Interaction Using Information-Driven Fuzzy Friend-Q Learning.

Estimating Emotional Intensity from Body Poses for Human-Robot Interaction

Dynamic Emotion Understanding in Human–Robot Interaction Based on Two-Layer Fuzzy SVR-TS Model

A Facial Expression Emotion Recognition Based Human-robot Interaction System

Emotional States Based 3-D Fuzzy Atmosfield for Casual Communication Between Humans and Robots

A Multimodal Emotional Communication Based Humans-Robots Interaction System

Human-Robot Emotional Interaction Model Based on Reinforcement Learning

Communication Atmosphere in Humans and Robots Interaction Based on Fuzzy Analytical Hierarchy Process

Research on Emotion Body Language Model of the Humanoid Robot

Human-Robot Interaction by Understanding Upper Body Gestures

Research and Implementation of Human-Computer Interaction System Based on Human Body Attitude Recognition Algorithm

Analysis of Influencing Factors on Humanoid Robots' Emotion Expressions by Body Language.

An Emotion Recognition Method for Humanoid Robot Body Movements Based on a PSO-BP-RMSProp Neural Network

ARBEE: Towards Automated Recognition of Bodily Expression of Emotion In the Wild

Emotion-Age-Gender-Nationality Based Intention Understanding in Human–Robot Interaction Using Two-Layer Fuzzy Support Vector Regression