Abstract:Reliably planning fingertip grasps for multi-fingered hands lies as a key challenge for many tasks including tool use, insertion, and dexterous in-hand manipulation. This task becomes even more difficult when the robot lacks an accurate model of the object to be grasped. Tactile sensing offers a promising approach to account for uncertainties in object shape. However, current robotic hands tend to lack full tactile coverage. As such, a problem arises of how to plan and execute grasps for multi-fingered hands such that contact is made with the area covered by the tactile sensors. To address this issue, we propose an approach to grasp planning that explicitly reasons about where the fingertips should contact the estimated object surface while maximizing the probability of grasp success. Key to our method's success is the use of visual surface estimation for initial planning to encode the contact constraint. The robot then executes this plan using a tactile-feedback controller that enables the robot to adapt to online estimates of the object's surface to correct for errors in the initial plan. Importantly, the robot never explicitly integrates object pose or surface estimates between visual and tactile sensing, instead it uses the two modalities in complementary ways. Vision guides the robots motion prior to contact; touch updates the plan when contact occurs differently than predicted from vision. We show that our method successfully synthesises and executes precision grasps for previously unseen objects using surface estimates from a single camera view. Further, our approach outperforms a state of the art multi-fingered grasp planner, while also beating several baselines we propose.

DiPGrasp: Parallel Local Searching for Efficient Differentiable Grasp Planning

A Cascaded Deep Learning Framework for Real-time and Robust Grasp Planning

TPGP: Temporal-Parametric Optimization with Deep Grasp Prior for Dexterous Motion Planning

Work Day and Night: A Learning Based Illumination Irrelevant Grasp Planning Method

Robot Grasp Planning: A Learning from Demonstration-Based Approach

Reach-to-Grasp Planning for a Synergy-Controlled Robotic Hand Based on Grasp Quality Prediction

Multifingered Grasp Planning Based on Gaussian Process Implicit Surface and its Partial Differentials

A Cross Complementarity Method for Simultaneous End-Effector Selection and Grasp Planning

Attention Based Visual Analysis for Fast Grasp Planning with a Multi-Fingered Robotic Hand

Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection

Robot Grasp Planning Based on Demonstrated Grasp Strategies

An Adaptive Planning Framework for Dexterous Robotic Grasping with Grasp Type Detection

Video Driven Adaptive Grasp Planning of Virtual Hand Using Deep Reinforcement Learning

Grasp Planning Via Hand-Object Geometric Fitting

Grasp Planning for Flexible Production with Small Lot Sizes based on CAD models using GPIS and Bayesian Optimization

SpringGrasp: Synthesizing Compliant, Dexterous Grasps under Shape Uncertainty

Transferring Grasping Across Grippers: Learning–Optimization Hybrid Framework for Generalized Planar Grasp Generation

EfficientGrasp: A Unified Data-Efficient Learning to Grasp Method for Multi-Fingered Robot Hands

Differentiable Robot Neural Distance Function for Adaptive Grasp Synthesis on a Unified Robotic Arm-Hand System

Grasp planning and implementation for dexterous hands by human demonstration

Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch