Abstract:Social robotics is an emerging area that is becoming present in social spaces, by introducing autonomous social robots. Social robots offer services, perform tasks, and interact with people in such social environments, demanding more efficient and complex Human-Robot Interaction (HRI) designs. A strategy to improve HRI is to provide robots with the capacity of detecting the emotions of the people around them to plan a trajectory, modify their behaviour, and generate an appropriate interaction with people based on the analysed information. However, in social environments in which it is common to find a group of persons, new approaches are needed in order to make robots able to recognise groups of people and the emotion of the groups, which can be also associated with a scene in which the group is participating. Some existing studies are focused on detecting group cohesion and the recognition of group emotions; nevertheless, these works do not focus on performing the recognition tasks from a robocentric perspective, considering the sensory capacity of robots. In this context, a system to recognise scenes in terms of groups of people, to then detect global (prevailing) emotions in a scene, is presented. The approach proposed to visualise and recognise emotions in typical HRI is based on the face size of people recognised by the robot during its navigation (face sizes decrease when the robot moves away from a group of people). On each frame of the video stream of the visual sensor, individual emotions are recognised based on the Visual Geometry Group (VGG) neural network pre-trained to recognise faces (VGGFace); then, to detect the emotion of the frame, individual emotions are aggregated with a fusion method, and consequently, to detect global (prevalent) emotion in the scene (group of people), the emotions of its constituent frames are also aggregated. Additionally, this work proposes a strategy to create datasets with images/videos in order to validate the estimation of emotions in scenes and personal emotions. Both datasets are generated in a simulated environment based on the Robot Operating System (ROS) from videos captured by robots through their sensory capabilities. Tests are performed in two simulated environments in ROS/Gazebo: a museum and a cafeteria. Results show that the accuracy in the detection of individual emotions is 99.79% and the detection of group emotion (scene emotion) in each frame is 90.84% and 89.78% in the cafeteria and the museum scenarios, respectively.

Environment Recognition Using Robot Camera

Cgan Based Facial Expression Recognition for Human-Robot Interaction

A Facial Expression Emotion Recognition Based Human-robot Interaction System

Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

Emotion Recognition via Environmental Context and Human Body

Intelligent Appearance and shape based facial emotion recognition for a humanoid robot

Adaptive Facial Point Detection and Emotion Recognition for a Humanoid Robot

Design of Service Robot Based on User Emotion Recognition and Environmental Monitoring

Dynamic Emotion Understanding in Human–Robot Interaction Based on Two-Layer Fuzzy SVR-TS Model

Emotion recognition models for companion robots

Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization

Intelligent Facial Emotion Recognition and Semantic-Based Topic Detection for A Humanoid Robot

A Multimodal Emotional Communication Based Humans-Robots Interaction System

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Multimodal Facial Emotion Recognition Using Improved Convolution Neural Networks Model

Group Emotion Detection Based on Social Robot Perception

Scene Recognition Mechanism for Service Robot Adapting Various Families: A CNN-based Approach Using Multi-Type Cameras

Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

A hybrid deep learning neural approach for emotion recognition from facial expressions for socially assistive robots

Facial Expressions Recognition for Human-Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer

Human Face Emotion in 3D Using Machine Learning