Abstract:Reliably planning fingertip grasps for multi-fingered hands lies as a key challenge for many tasks including tool use, insertion, and dexterous in-hand manipulation. This task becomes even more difficult when the robot lacks an accurate model of the object to be grasped. Tactile sensing offers a promising approach to account for uncertainties in object shape. However, current robotic hands tend to lack full tactile coverage. As such, a problem arises of how to plan and execute grasps for multi-fingered hands such that contact is made with the area covered by the tactile sensors. To address this issue, we propose an approach to grasp planning that explicitly reasons about where the fingertips should contact the estimated object surface while maximizing the probability of grasp success. Key to our method's success is the use of visual surface estimation for initial planning to encode the contact constraint. The robot then executes this plan using a tactile-feedback controller that enables the robot to adapt to online estimates of the object's surface to correct for errors in the initial plan. Importantly, the robot never explicitly integrates object pose or surface estimates between visual and tactile sensing, instead it uses the two modalities in complementary ways. Vision guides the robots motion prior to contact; touch updates the plan when contact occurs differently than predicted from vision. We show that our method successfully synthesises and executes precision grasps for previously unseen objects using surface estimates from a single camera view. Further, our approach outperforms a state of the art multi-fingered grasp planner, while also beating several baselines we propose.

Grasping Using Tactile Sensing and Deep Calibration

Automatic Grasping Using Tactile Sensing and Deep Calibration

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Adaptive Grasping of the Soft Prosthetic Hand Via Tactile Feedback

More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch

Humanoid Robot Grasping with a Soft Gripper Through a Learned Inverse Model of a Central Pattern Generator and Tactile Servoing

Grasping Control for a Neuro-Cognitive Robot.

Tactile Glove-Decode and Classify the Human Grasping Process

Grasping Force Control of Multi-Fingered Robotic Hands Through Tactile Sensing for Object Stabilization.

A Comprehensive Review of Robot Intelligent Grasping Based on Tactile Perception.

Learning Gentle Grasping from Human-Free Force Control Demonstration

T-TD3: A Reinforcement Learning Framework for Stable Grasping of Deformable Objects Using Tactile Prior

An Autonomous Grasping Control System Based on Visual Object Recognition and Tactile Perception

Fine Manipulation Using a Tactile Skin: Learning in Simulation and Sim-to-Real Transfer

Tactile-Driven Non-Prehensile Object Manipulation via Extrinsic Contact Mode Control

Tactile sensor-less fingertip contact detection and force estimation for stable grasping with an under-actuated hand

Autonomous Robotic Manipulation: Real-Time, Deep-Learning Approach for Grasping of Unknown Objects

Robotic grasping using visual and tactile sensing

Predict Robot Grasp Outcomes Based On Multi-Modal Information

Learning Fine Pinch-Grasp Skills using Tactile Sensing from A Few Real-world Demonstrations

Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch