Abstract:Simulation is widely used in robotics for system verification and large-scale data collection. However, simulating sensors, including tactile sensors, has been a long-standing challenge. In this paper, we propose Taxim, a realistic and high-speed simulation model for a vision-based tactile sensor, GelSight. A GelSight sensor uses a piece of soft elastomer as the medium of contact and embeds optical structures to capture the deformation of the elastomer, which infers the geometry and forces applied at the contact surface. We propose an example-based method for simulating GelSight: we simulate the optical response to the deformation with a polynomial look-up table. This table maps the deformed geometries to pixel intensity sampled by the embedded camera. In order to simulate the surface markers' motion that is caused by the surface stretch of the elastomer, we apply the linear elastic deformation theory and the superposition principle. The simulation model is calibrated with less than 100 data points from a real sensor. The example-based approach enables the model to easily migrate to other GelSight sensors or its variations. To the best of our knowledge, our simulation framework is the first to incorporate marker motion field simulation that derives from elastomer deformation together with the optical simulation, creating a comprehensive and computationally efficient tactile simulation framework. Experiments reveal that our optical simulation has the lowest pixel-wise intensity errors compared to prior work and can run online with CPU computing. Our code and supplementary materials are open-sourced at <a class="link-external link-https" href="https://github.com/CMURoboTouch/Taxim" rel="external noopener nofollow">this https URL</a>.

Elastic Tactile Simulation Towards Tactile-Visual Perception.

Elastic Interaction of Particles for Robotic Tactile Simulation

TacIPC: Intersection- and Inversion-free FEM-based Elastomer Simulation For Optical Tactile Sensors

Design, Optimization and Application of Tactile Sensor Based on Virtual Binocular Vision

Vision-based Tactile Image Generation via Contact Condition-guided Diffusion Model

DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation

A Large-Area Flexible Tactile Sensor for Multi-Touch and Force Detection Using Electrical Impedance Tomography

Taxim: An Example-based Simulation Model for GelSight Tactile Sensors

Large‐Area Hand‐Covering Elastomeric Electronic Skin Sensor with Distributed Multifunctional Sensing Capability

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Flexible Tactile Sensor Array for Distributed Tactile Sensing and Slip Detection in Robotic Hand Grasping

Data-efficient Tactile Sensing with Electrical Impedance Tomography

Progress on Flexible Tactile Sensors in Robotic Applications on Objects Properties Recognition, Manipulation and Human-Machine Interactions

Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors

Electrical Impedance Tomography-Based Electronic Skin for Multi-Touch Tactile Sensing Using Hydrogel Material and FISTA Algorithm

Sim-to-Real for High-Resolution Optical Tactile Sensing: From Images to Three-Dimensional Contact Force Distributions

Acquisition and Prediction of High-Density Tactile Field Data for Rigid and Flexible Objects

Simulation of Vision-based Tactile Sensors with Efficiency-tunable Rendering

ELTac: A Vision-Based Electroluminescent Tactile Sensing Skin for Force Localization and Magnitude Estimation

Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing

Crossing the Reality Gap in Tactile-Based Learning