Abstract:When a multi-fingered dexterous hand interacts with the external environment, it encounters various challenges, including the utilization of complex control techniques and the intricate coordination of finger motion sequences. Previous studies have primarily concentrated on investigating the interaction between multi-fingered dexterous hands and external objects, usually using model-based control or model-free reinforcement learning techniques. However, during practical implementation, various constraining factors are encountered, such as intricate modeling and limited interaction capabilities. In practical scenarios, the utilization of multi-fingered dexterous hands is imperative for the swift and efficient execution of a wide range of interactive tasks, including but not limited to throwing a ball and playing rock-paper-scissors. These tasks require skilled manual dexterity to demonstrate both precise control and quick responsiveness. To tackle this issue, we propose a hierarchical control approach for multi-fingered dexterous hands with interactive functionalities, utilizing model-free deep reinforcement learning. The complex interaction task is decomposed into simple sub-tasks using hierarchical strategy and action primitive decomposition, which effectively reduces the complexity of the action space, and achieves the motion planning and end finger trajectory control of dexterous hand. In a simulated environment, the aforementioned method has successfully executed interactive tasks, including ball throwing and playing rock-paper-scissors. It achieved a maximum normalized reward of 0.83 and an 84 % success rate. These results are noteworthy in terms of both control accuracy and response speed. This study offers novel insights into the effective resolution of the intricate challenges associated with interactions involving multi-fingered dexterous hands and human-computer interaction.

Dynamic Handover: Throw and Catch with Bimanual Hands

A Human-Robot Collaboration System for Object Handover

Catch It! Learning to Catch in Flight with Mobile Dexterous Hands

Advancing robots with greater dynamic dexterity: A large-scale multi-view and multi-modal dataset of human-human throw&catch of arbitrary objects

Learning Dynamic Robot-to-Human Object Handover from Human Feedback

Learning Human-to-Robot Dexterous Handovers for Anthropomorphic Hand

DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands

Whole-Body Dynamic Throwing with Legged Manipulators

GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation, Demonstration, and Imitation

Dynamic Throwing with Robotic Material Handling Machines

Probabilistic Motion Prediction and Skill Learning for Human-to-Cobot Dual-Arm Handover Control

Fast and Comfortable Robot-to-Human Handover for Mobile Cooperation Robot System

Interactive learning for multi-finger dexterous hand: A model-free hierarchical deep reinforcement learning approach

ContactHandover: Contact-Guided Robot-to-Human Object Handover

Multi-fingered Dynamic Grasping for Unknown Objects

Throwing Objects into A Moving Basket While Avoiding Obstacles

TossingBot: Learning to Throw Arbitrary Objects With Residual Physics

Dynamic Hand Gesture-Featured Human Motor Adaptation in Tool Delivery using Voice Recognition

Harnessing the Synergy between Pushing, Grasping, and Throwing to Enhance Object Manipulation in Cluttered Scenarios

Object-Centric Dexterous Manipulation from Human Motion Data

Grasping Diverse Objects with Simulated Humanoids