Abstract:Achieving human-level dexterity in robotics remains a critical open problem. Even simple dexterous manipulation tasks pose significant difficulties due to the high number of degrees of freedom and the need for cooperation among heterogeneous agents (e.g., finger joints). While some researchers have utilized reinforcement learning (RL) to control a single hand in manipulating objects, tasks that require coordinated bimanual cooperation are still under-explored due to the fewer suitable environments, which can result in difficulties and sub-optimal performance. To address these challenges, we introduce Bi-DexHands, a simulator with two dexterous hands featuring 20 bimanual manipulation tasks and thousands of target objects, designed to match various levels of human motor skills based on cognitive science research. We developed Bi-DexHands in Issac Gym, enabling highly efficient RL training at over 30,000 frames per second using a single NVIDIA RTX 3090. Based on Bi-DexHands, we present a comprehensive evaluation of popular RL algorithms in different settings, including single-agent/multi-agent RL, offline RL, multi-task RL, and meta RL. Our findings show that on-policy algorithms, such as PPO, can master simple manipulation tasks that correspond to those of 48-month-old babies, such as catching a flying object or opening a bottle. Furthermore, multi-agent RL can improve the ability to perform manipulations that require skilled bimanual cooperation, such as lifting a pot or stacking blocks. Despite achieving success in individual tasks, current RL algorithms struggle to learn multiple manipulation skills in most multi-task and few-shot learning scenarios. This highlights the need for further research and development within the RL community.

DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation

Disentangled Attention As Intrinsic Regularization for Bimanual Multi-Object Manipulation

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning

Bi-DexHands: Towards Human-Level Bimanual Dexterous Manipulation

InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation

Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrations

Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning

Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost

Stabilize to Act: Learning to Coordinate for Bimanual Manipulation

Enabling Versatility and Dexterity of the Dual-Arm Manipulators: A General Framework Toward Universal Cooperative Manipulation

Efficient Bimanual Handover and Rearrangement Via Symmetry-Aware Actor-Critic Learning.

Cross-Embodiment Dexterous Grasping with Reinforcement Learning

Discovering Synergies for Robot Manipulation with Multi-Task Reinforcement Learning

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

A High-Efficient Reinforcement Learning Approach for Dexterous Manipulation

A Multi-Agent Approach for Adaptive Finger Cooperation in Learning-based In-Hand Manipulation

Bi-Touch: Bimanual Tactile Manipulation With Sim-to-Real Deep Reinforcement Learning

Grasp with Push Policy for Multi-Finger Dexterity Hand Based on Deep Reinforcement Learning☆

Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation