Abstract:Anthropomorphic robot avatars present a conceptually novel approach to remote affective communication, allowing people across the world a wider specter of emotional and social exchanges over traditional 2D and 3D image data. However, there are several limitations of current telepresence robots, such as the high weight, complexity of the system that prevents its fast deployment, and the limited workspace of the avatars mounted on either static or wheeled mobile platforms. In this paper, we present a novel concept of telecommunication through a robot avatar based on an anthropomorphic swarm of drones; SwarMan. The developed system consists of nine nanocopters controlled remotely by the operator through a gesture recognition interface. SwarMan allows operators to communicate by directly following their motions and by recognizing one of the prerecorded emotional patterns, thus rendering the captured emotion as illumination on the drones. The LSTM MediaPipe network was trained on a collected dataset of 600 short videos with five emotional gestures. The accuracy of achieved emotion recognition was 97% on the test dataset. As communication through the swarm avatar significantly changes the visual appearance of the operator, we investigated the ability of the users to recognize and respond to emotions performed by the swarm of drones. The experimental results revealed a high consistency between the users in rating emotions. Additionally, users indicated low physical demand (2.25 on the Likert scale) and were satisfied with their performance (1.38 on the Likert scale) when communicating by the SwarMan interface.

Social Interaction with Drones using Human Emotion Recognition

Emotion Appropriateness in Human–Drone Interaction

Towards Affective Drone Swarms: A Preliminary Crowd-Sourced Study

Evaluating the Potential of Drone Swarms in Nonverbal HRI Communication

Anthropomorphism and Affective Perception: Dimensions, Measurements, and Interdependencies in Aerial Robotics

SwarMan: Anthropomorphic Swarm of Drones Avatar with Body Tracking and Deep Learning-Based Gesture Recognition

Emotional States Based 3-D Fuzzy Atmosfield for Casual Communication Between Humans and Robots

Teaching Drones on the Fly: Can Emotional Feedback Serve as Learning Signal for Training Artificial Agents?

Personalized Human-Swarm Interaction through Hand Motion

Perception of Human Interaction Based on Motion Trajectories: From Aerial Videos to Decontextualized Animations.

Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

Conceptual Design of Human-Drone Communication in Collaborative Environments

Co-existing with Drones: A Virtual Exploration of Proxemic Behaviours and Users' Insights on Social Drones

A Multimodal Emotional Communication Based Humans-Robots Interaction System

Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

DroneLight: Drone Draws in the Air using Long Exposure Light Painting and ML

A MultiModal Social Robot Toward Personalized Emotion Interaction

Multi-Modal Based Fuzzy Atmosfield in Human-Robot Interaction

From Motions to Emotions: Can the Fundamental Emotions be Expressed in a Robot Swarm?

Intuitive Human-Robot Interface: A 3-Dimensional Action Recognition and UAV Collaboration Framework