Abstract:This work explores conditions under which multi-finger grasping algorithms can attain robust sim-to-real transfer. While numerous large datasets facilitate learning generative models for multi-finger grasping at scale, reliable real-world dexterous grasping remains challenging, with most methods degrading when deployed on hardware. An alternate strategy is to use discriminative grasp evaluation models for grasp selection and refinement, conditioned on real-world sensor measurements. This paradigm has produced state-of-the-art results for vision-based parallel-jaw grasping, but remains unproven in the multi-finger setting. In this work, we find that existing datasets and methods have been insufficient for training discriminitive models for multi-finger grasping. To train grasp evaluators at scale, datasets must provide on the order of millions of grasps, including both positive and negative examples, with corresponding visual data resembling measurements at inference time. To that end, we release a new, open-source dataset of 3.5M grasps on 4.3K objects annotated with RGB images, point clouds, and trained NeRFs. Leveraging this dataset, we train vision-based grasp evaluators that outperform both analytic and generative modeling-based baselines on extensive simulated and real-world trials across a diverse range of objects. We show via numerous ablations that the key factor for performance is indeed the evaluator, and that its quality degrades as the dataset shrinks, demonstrating the importance of our new dataset. Project website at: <a class="link-external link-https" href="https://sites.google.com/view/get-a-grip-dataset" rel="external noopener nofollow">this https URL</a>.

DA$^{2}$ Dataset: Toward Dexterity-Aware Dual-Arm Grasping.

Bimanual Grasp Synthesis for Dexterous Robot Hands

DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation

RealDex: Towards Human-like Grasping for Robotic Dexterous Hand

MultiGripperGrasp: A Dataset for Robotic Grasping from Parallel Jaw Grippers to Dexterous Hands

Multi-task real-robot data with gaze attention for dual-arm fine manipulation

DexFuncGrasp: A Robotic Dexterous Functional Grasp Dataset Constructed from a Cost-Effective Real-Simulation Annotation System

DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulation

Grasp as You Say: Language-guided Dexterous Grasp Generation

GenDexGrasp: Generalizable Dexterous Grasping

DexYCB: A Benchmark for Capturing Hand Grasping of Objects

DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenes

Get a Grip: Multi-Finger Grasp Evaluation at Scale Enables Robust Sim-to-Real Transfer

BODex: Scalable and Efficient Robotic Dexterous Grasp Synthesis Using Bilevel Optimization

DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrations

Toward Human-Like Grasp: Dexterous Grasping via Semantic Representation of Object-Hand

Robust grasping across diverse sensor qualities: The GraspNet-1Billion dataset

DVGG: Deep Variational Grasp Generation for Dextrous Manipulation

Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation

QDGset: A Large Scale Grasping Dataset Generated with Quality-Diversity

D(R, O) Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping