Abstract:We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In contrast to prior tactile simulators which primarily focus on manipulating rigid bodies and often rely on simplified approximations to model stress and deformations of materials in contact, DIFFTACTILE emphasizes physics-based contact modeling with high fidelity, supporting simulations of diverse contact modes and interactions with objects possessing a wide range of material properties. Our system incorporates several key components, including a Finite Element Method (FEM)-based soft body model for simulating the sensing elastomer, a multi-material simulator for modeling diverse object types (such as elastic, elastoplastic, cables) under manipulation, a penalty-based contact model for handling contact dynamics. The differentiable nature of our system facilitates gradient-based optimization for both 1) refining physical properties in simulation using real-world data, hence narrowing the sim-to-real gap and 2) efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, we introduce a method to infer the optical response of our tactile sensor to contact using an efficient pixel-based neural module. We anticipate that DIFFTACTILE will serve as a useful platform for studying contact-rich manipulations, leveraging the benefits of dense tactile feedback and differentiable physics. Code and supplementary materials are available at the project website <a class="link-external link-https" href="https://difftactile.github.io/" rel="external noopener nofollow">this https URL</a>.

Acquisition and Prediction of High-Density Tactile Field Data for Rigid and Flexible Objects

Beyond End-Effector: Utilizing High-Resolution Tactile Signals for Physical Human–Robot Interaction

Progress on Flexible Tactile Sensors in Robotic Applications on Objects Properties Recognition, Manipulation and Human-Machine Interactions

Development of Flexible Tactile Sensor for the Envelop of Curved Robotic Hand Finger in Grasping Force Sensing

A Novel Inverse Solution Of Contact Force Based On A Sparse Tactile Sensor Array

Flexible Tactile Sensor Array for Distributed Tactile Sensing and Slip Detection in Robotic Hand Grasping

9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation

Robot Tactile Sensing: Vision Based Tactile Sensor for Force Perception

Interpreting and Predicting Tactile Signals for the SynTouch BioTac

A Dual-Modal Vision-Based Tactile Sensor for Robotic Hand Grasping.

Vision-based Tactile Image Generation via Contact Condition-guided Diffusion Model

FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensors

Impact of Tactile Sensor Quantities and Placements on Learning-based Dexterous Manipulation

Robust Affordable 3D Haptic Sensation via Learning Deformation Patterns

A Large-Area Flexible Tactile Sensor for Multi-Touch and Force Detection Using Electrical Impedance Tomography

Ground truth force distribution for learning-based tactile sensing: a finite element approach

Robust Learning of Tactile Force Estimation through Robot Interaction

3D Dense Reconstruction of Vision-based Tactile Sensor with Coded Markers

Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor

DTactive: A Vision-Based Tactile Sensor with Active Surface

DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation