Abstract:Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

Gesture based master-slave controller for a social humanoid robot

Modular Development of Master-Slave Asymmetric Teleoperation Systems with a Novel Workspace Mapping Algorithm

Inverse Kinematics and Master-Slave Control for a 7-DoF Tendon-Driven Humanoid Robot Arm

An Iterative Actuation Method for Master-Slave Telemanipulation by Humanoid Robot Arm

Research on master-slave teleoperation control algorithm based on exoskeleton master hand

An IMUs and potentiometer-based controller for robotic arm-hand teleoperation

High Speed Whole Body Dynamic Motion Experiment with Real Time Master-Slave Humanoid Robot System

Master Manipulator Designed For Highly Articulated Robotic Instruments In Single Access Surgery

Operator Attitude Algorithm for an Isomorphic Master-slave Telerobotic System

Hybrid Mapping Method: from Human to Robotic Hands with Dissimilar Kinematics

A Self-controlled Master-Slave Robot and Its Application for Upper Limb Rehabilitation

Hold My Hand: Development of a Force Controller and System Architecture for Joint Walking with a Companion Robot

Deep learning-based control framework for dynamic contact processes in humanoid grasping

Human-robot Shared Control for Humanoid Manipulator Trajectory Planning

Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots

A heterogeneous master-slave teleoperation method for 7-DOF manipulator

Biomimetic learning of hand gestures in a humanoid robot

A Novel Grasping Teleoperation Control for Dual-Handed System

A Handheld Master Controller for Robot-Assisted Microsurgery

Dual-Master/Single-Slave Haptic Teleoperation System for Semiautonomous Bilateral Control of Hexapod Robot Subject to Deformable Rough Terrain