Abstract:Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

Learning,Generating and Adapting Wave Gestures for Expressive Human-Robot Interaction

EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning

Toward Automated Generation of Affective Gestures from Text:A Theory-Driven Approach

Imitating by Generating: Deep Generative Models for Imitation of Interactive Tasks

Learning to gesticulate by observation using a deep generative approach

Biomimetic learning of hand gestures in a humanoid robot

Learning to generate pointing gestures in situated embodied conversational agents

A Generative Model to Embed Human Expressivity into Robot Motions

Modeling and evaluating beat gestures for social robots

HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitation

Learning Responsive Humanoid Motion Skills from Graph-Powered Motion Matching

Communicating human intent to a robotic companion by multi-type gesture sentences

Learning Multimodal Latent Dynamics for Human-Robot Interaction

An Approach to Elicit Human-Understandable Robot Expressions to Support Human-Robot Interaction

Co-Speech Gesture Synthesis using Discrete Gesture Token Learning

Learning Physical Collaborative Robot Behaviors From Human Demonstrations

Hands-On Robotics: Enabling Communication Through Direct Gesture Control

Interactive Robot Learning of Gestures, Language and Affordances

Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation

Gesticulator: A framework for semantically-aware speech-driven gesture generation

Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents