Abstract:In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored perception model in simulation that estimates a probability distribution over possible object poses given a tactile observation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor, we match it against the pre-computed set using an object-specific embedding learned using contrastive learning. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB tactile observations to binary contact shapes. This mapping, which can be reused across object and sensor instances, is the only step trained with real sensor data. This results in a perception model that localizes objects from the first real tactile observation. Importantly, it produces pose distributions and can incorporate additional pose constraints coming from other perception systems, contacts, or priors. We provide quantitative results for 20 objects. Tac2Pose provides high accuracy pose estimations from distinctive tactile observations while regressing meaningful pose distributions to account for those contact shapes that could result from different object poses. We also test Tac2Pose on object models reconstructed from a 3D scanner, to evaluate the robustness to uncertainty in the object model. Finally, we demonstrate the advantages of Tac2Pose compared with three baseline methods for tactile pose estimation: directly regressing the object pose with a neural network, matching an observed contact to a set of possible contacts using a standard classification neural network, and direct pixel comparison of an observed contact with a set of possible contacts. Website: <a class="link-external link-http" href="http://mcube.mit.edu/research/tac2pose.html" rel="external noopener nofollow">this http URL</a>

Pose-Based Tactile Servoing: Controlled Soft Touch Using Deep Learning

Pose and shear-based tactile servoing

Pose-and-shear-based tactile servoing

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Optimal Deep Learning for Robot Touch

Bi-Touch: Bimanual Tactile Manipulation With Sim-to-Real Deep Reinforcement Learning

DotTip: Enhancing Dexterous Robotic Manipulation with a Tactile Fingertip Featuring Curved Perceptual Morphology

Learning Self-Supervised Representations from Vision and Touch for Active Sliding Perception of Deformable Surfaces

Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control

Use of tactile feedback to control exploratory movements to characterize object compliance

DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexterity

Touch and deformation perception of soft manipulators with capacitive e-skins and deep learning

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Robust Learning of Tactile Force Estimation through Robot Interaction

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments

Improving Soft-Capture Phase Success in Space Debris Removal Missions: Leveraging Deep Reinforcement Learning and Tactile Feedback

DTactive: A Vision-Based Tactile Sensor with Active Surface

Tac2Pose: Tactile Object Pose Estimation from the First Touch

Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation

A Low-Cost, Easy-to-Manufacture, Flexible, Multi-Taxel Tactile Sensor and its Application to In-Hand Object Recognition

Tactile-Driven Gentle Grasping for Human-Robot Collaborative Tasks