Abstract:We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In contrast to prior tactile simulators which primarily focus on manipulating rigid bodies and often rely on simplified approximations to model stress and deformations of materials in contact, DIFFTACTILE emphasizes physics-based contact modeling with high fidelity, supporting simulations of diverse contact modes and interactions with objects possessing a wide range of material properties. Our system incorporates several key components, including a Finite Element Method (FEM)-based soft body model for simulating the sensing elastomer, a multi-material simulator for modeling diverse object types (such as elastic, elastoplastic, cables) under manipulation, a penalty-based contact model for handling contact dynamics. The differentiable nature of our system facilitates gradient-based optimization for both 1) refining physical properties in simulation using real-world data, hence narrowing the sim-to-real gap and 2) efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, we introduce a method to infer the optical response of our tactile sensor to contact using an efficient pixel-based neural module. We anticipate that DIFFTACTILE will serve as a useful platform for studying contact-rich manipulations, leveraging the benefits of dense tactile feedback and differentiable physics. Code and supplementary materials are available at the project website <a class="link-external link-https" href="https://difftactile.github.io/" rel="external noopener nofollow">this https URL</a>.

Tactile rendering based on skin stress optimization

Data Analysis on Virtual Stiffness in 6dofs Haptic Rendering System

Tactile rendering of 3D features on touch surfaces

Surface haptic rendering of virtual shapes through change in surface temperature

Displaying Sensed Tactile Cues with a Fingertip Haptic Device

Tactile Weight Rendering: A Review for Researchers and Developers

How to Make the Skin Contact Area Controllable by Optical Calibration in Wearable Tactile Displays of Softness

DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation

Tactile Neural De-rendering

Design, Motivation and Evaluation of a Full-Resolution Optical Tactile Sensor

Haptic Sketches on the Arm for Manipulation in Virtual Reality

Simulation of Vision-based Tactile Sensors with Efficiency-tunable Rendering

Design and evaluation of a multi-finger skin-stretch tactile interface for hand rehabilitation robots

Sim-to-Real for High-Resolution Optical Tactile Sensing: From Images to Three-Dimensional Contact Force Distributions

Realism of Tactile Texture Playback: A Combination of Stretch and Vibration

A Multi-Point Cutaneous Tactile Device for Fingertip Haptics

Optimizing Multi-Touch Textile and Tactile Skin Sensing Through Circuit Parameter Estimation

A bioinspired tactile scanner for computer haptics

Design, Optimization and Application of Tactile Sensor Based on Virtual Binocular Vision

Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering

Encoding of tactile information in hand via skin-integrated wireless haptic interface