Abstract:Bimanual dexterous manipulation is a critical yet underexplored area in robotics. Its high-dimensional action space and inherent task complexity present significant challenges for policy learning, and the limited task diversity in existing benchmarks hinders general-purpose skill development. Existing approaches largely depend on reinforcement learning, often constrained by intricately designed reward functions tailored to a narrow set of tasks. In this work, we present a novel approach for efficiently learning diverse bimanual dexterous skills from abundant human demonstrations. Specifically, we introduce BiDexHD, a framework that unifies task construction from existing bimanual datasets and employs teacher-student policy learning to address all tasks. The teacher learns state-based policies using a general two-stage reward function across tasks with shared behaviors, while the student distills the learned multi-task policies into a vision-based policy. With BiDexHD, scalable learning of numerous bimanual dexterous skills from auto-constructed tasks becomes feasible, offering promising advances toward universal bimanual dexterous manipulation. Our empirical evaluation on the TACO dataset, spanning 141 tasks across six categories, demonstrates a task fulfillment rate of 74.59% on trained tasks and 51.07% on unseen tasks, showcasing the effectiveness and competitive zero-shot generalization capabilities of BiDexHD. For videos and more information, visit our project page <a class="link-external link-https" href="https://sites.google.com/view/bidexhd" rel="external noopener nofollow">this https URL</a>.

Learning Task Priorities from Demonstrations

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

The Art of Imitation: Learning Long-Horizon Manipulation Tasks from Few Demonstrations

Task-Parameterized Imitation Learning with Time-Sensitive Constraints

Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrations

Interactive Human-in-the-loop Coordination of Manipulation Skills Learned from Demonstration

Hierarchical and Parameterized Learning of Pick-and-place Manipulation from Under-Specified Human Demonstrations

A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations

Learning Spatial Bimanual Action Models Based on Affordance Regions and Human Demonstrations

Learning a Universal Human Prior for Dexterous Manipulation from Human Preference

Learning Periodic Tasks from Human Demonstrations

Learning Under-Specified Object Manipulations from Human Demonstrations.

Learning from demonstration with model-based Gaussian process

Multi-Phase Multi-Objective Dexterous Manipulation with Adaptive Hierarchical Curriculum

A Task Learning Mechanism for the Telerobots

Towards Transferring Skills to Flexible Surgical Robots with Programming by Demonstration and Reinforcement Learning

Learning Manipulation Tasks in Dynamic and Shared 3D Spaces

Learning Task-Parameterized Skills from Few Demonstrations

Learning Generalizable 3D Manipulation With 10 Demonstrations

BiRP: Learning Robot Generalized Bimanual Coordination using Relative Parameterization Method on Human Demonstration