Abstract:Simulation is widely used in robotics for system verification and large-scale data collection. However, simulating sensors, including tactile sensors, has been a long-standing challenge. In this paper, we propose Taxim, a realistic and high-speed simulation model for a vision-based tactile sensor, GelSight. A GelSight sensor uses a piece of soft elastomer as the medium of contact and embeds optical structures to capture the deformation of the elastomer, which infers the geometry and forces applied at the contact surface. We propose an example-based method for simulating GelSight: we simulate the optical response to the deformation with a polynomial look-up table. This table maps the deformed geometries to pixel intensity sampled by the embedded camera. In order to simulate the surface markers' motion that is caused by the surface stretch of the elastomer, we apply the linear elastic deformation theory and the superposition principle. The simulation model is calibrated with less than 100 data points from a real sensor. The example-based approach enables the model to easily migrate to other GelSight sensors or its variations. To the best of our knowledge, our simulation framework is the first to incorporate marker motion field simulation that derives from elastomer deformation together with the optical simulation, creating a comprehensive and computationally efficient tactile simulation framework. Experiments reveal that our optical simulation has the lowest pixel-wise intensity errors compared to prior work and can run online with CPU computing. Our code and supplementary materials are open-sourced at <a class="link-external link-https" href="https://github.com/CMURoboTouch/Taxim" rel="external noopener nofollow">this https URL</a>.

Generation of GelSight Tactile Images for Sim2Real Learning

Beyond Flat GelSight Sensors: Simulation of Optical Tactile Sensors of Complex Morphologies for Sim2Real Learning

Bidirectional Sim-to-Real Transfer for GelSight Tactile Sensors With CycleGAN

Augmenting Tactile Simulators with Real-like and Zero-Shot Capabilities

Taxim: An Example-based Simulation Model for GelSight Tactile Sensors

GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force

TacEx: GelSight Tactile Simulation in Isaac Sim -- Combining Soft-Body and Visuotactile Simulators

Sim-to-Real for High-Resolution Optical Tactile Sensing: From Images to Three-Dimensional Contact Force Distributions

GelSlim 4.0: Focusing on Touch and Reproducibility

GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Deep Tactile Experience: Estimating Tactile Sensor Output from Depth Sensor Data

Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing

Improved GelSight tactile sensor for measuring geometry and slip

Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning

Touch2Touch: Cross-Modal Tactile Generation for Object Manipulation

GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger

FOTS: A Fast Optical Tactile Simulator for Sim2Real Learning of Tactile-motor Robot Manipulation Skills

GelSight360: An Omnidirectional Camera-Based Tactile Sensor for Dexterous Robotic Manipulation

General-Purpose Sim2Real Protocol for Learning Contact-Rich Manipulation With Marker-Based Visuotactile Sensors

GelStereo Palm: a Novel Curved Visuotactile Sensor for 3D Geometry Sensing