Abstract:Reliably planning fingertip grasps for multi-fingered hands lies as a key challenge for many tasks including tool use, insertion, and dexterous in-hand manipulation. This task becomes even more difficult when the robot lacks an accurate model of the object to be grasped. Tactile sensing offers a promising approach to account for uncertainties in object shape. However, current robotic hands tend to lack full tactile coverage. As such, a problem arises of how to plan and execute grasps for multi-fingered hands such that contact is made with the area covered by the tactile sensors. To address this issue, we propose an approach to grasp planning that explicitly reasons about where the fingertips should contact the estimated object surface while maximizing the probability of grasp success. Key to our method's success is the use of visual surface estimation for initial planning to encode the contact constraint. The robot then executes this plan using a tactile-feedback controller that enables the robot to adapt to online estimates of the object's surface to correct for errors in the initial plan. Importantly, the robot never explicitly integrates object pose or surface estimates between visual and tactile sensing, instead it uses the two modalities in complementary ways. Vision guides the robots motion prior to contact; touch updates the plan when contact occurs differently than predicted from vision. We show that our method successfully synthesises and executes precision grasps for previously unseen objects using surface estimates from a single camera view. Further, our approach outperforms a state of the art multi-fingered grasp planner, while also beating several baselines we propose.

Unknown Objects Grasp Planning Using Force/Torque Measurements

Adaptive Grasping of the Soft Prosthetic Hand Via Tactile Feedback

Multi-fingered Dynamic Grasping for Unknown Objects

Development of Robotic Hand Tactile Sensing System for Distributed Contact Force Sensing in Robotic Dexterous Multimodal Grasping

Design And Use Of An Apparatus For Presenting Graspable Objects In 3d Workspace

Multifingered Grasp Planning Based on Gaussian Process Implicit Surface and its Partial Differentials

Center-of-Mass-based Robust Grasp Planning for Unknown Objects Using Tactile-Visual Sensors

A Fast Grasp Planning Algorithm for Humanoid Robot Hands

Learning to Grasp Familiar Objects Based on Experience

Development of Flexible Tactile Sensor for the Envelop of Curved Robotic Hand Finger in Grasping Force Sensing

Tactile sensor-less fingertip contact detection and force estimation for stable grasping with an under-actuated hand

A System of Robotic Grasping with Experience Acquisition.

Unknown Object Grasping for Assistive Robotics

Design of a Tactile Sensing Robotic Gripper and Its Grasping Method

Sliding Touch-based Exploration for Modeling Unknown Object Shape with Multi-fingered Hands

Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch

Integrated Grasping Controller Leveraging Optical Proximity Sensors for Simultaneous Contact, Impact Reduction, and Force Control

Learning-Free Grasping of Unknown Objects Using Hidden Superquadrics

PointGrasp: Point Cloud-based Grasping for Tendon-driven Soft Robotic Glove Applications

Multifingered Grasping Based on Multimodal Reinforcement Learning

Generalized Grasping for Mechanical Grippers for Unknown Objects with Partial Point Cloud Representations