Abstract:Vision-based tactile sensors, through high-resolution optical measurements, can effectively perceive the geometric shape of objects and the force information during the contact process, thus helping robots acquire higher-dimensional tactile data. Vision-based tactile sensor simulation supports the acquisition and understanding of tactile information without physical sensors by accurately capturing and analyzing contact behavior and physical properties. However, the complexity of contact dynamics and lighting modeling limits the accurate reproduction of real sensor responses in simulations, making it difficult to meet the needs of different sensor setups and affecting the reliability and effectiveness of strategy transfer to practical applications. In this letter, we propose a contact-condition guided diffusion model that maps RGB images of objects and contact force data to high-fidelity, detail-rich vision-based tactile sensor images. Evaluations show that the three-channel tactile images generated by this method achieve a 60.58% reduction in mean squared error and a 38.1% reduction in marker displacement error compared to existing approaches based on lighting model and mechanical model, validating the effectiveness of our approach. The method is successfully applied to various types of tactile vision sensors and can effectively generate corresponding tactile images under complex loads. Additionally, it demonstrates outstanding reconstruction of fine texture features of objects in a Montessori tactile board texture generation task.

Model-Based 3D Contact Geometry Perception for Visual Tactile Sensor

Vision-based Tactile Image Generation via Contact Condition-guided Diffusion Model

Design, Optimization and Application of Tactile Sensor Based on Virtual Binocular Vision

DTact: A Vision-Based Tactile Sensor That Measures High-Resolution 3D Geometry Directly from Darkness

DTact: A Vision-Based Tactile Sensor that Measures High-Resolution 3D Geometry Directly from Darkness

9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation

Multidimensional Tactile Sensor with a Thin Compound Eye-Inspired Imaging System

Measurement Of Three-Dimensional Deformation And Load Using Vision-Based Tactile Sensor

Robot Tactile Sensing: Vision Based Tactile Sensor for Force Perception

DelTact: A Vision-based Tactile Sensor Using Dense Color Pattern

Real-Time Reconstruction of 3D Tactile Motion Field Via Multi-Task Learning

3D Dense Reconstruction of Vision-based Tactile Sensor with Coded Markers

Real-Time Reconstruction of 3-D Tactile Motion Field via Multitask Learning

Contact Position Estimation for a Compact and Finger-shaped Tactile Sensor

StereoTac: a Novel Visuotactile Sensor that Combines Tactile Sensing with 3D Vision

Tac3D: A Novel Vision-based Tactile Sensor for Measuring Forces Distribution and Estimating Friction Coefficient Distribution

3-D Object Reconstruction Using Tactile Sensing for Multi-fingered Hand

A 3-D Surface Reconstruction with Shadow Processing for Optical Tactile Sensors

A Distributed Force Reconstruction Mapping Model with Physical Compactness and Low Computational Cost for Vision-based Tactile Sensors

Development of a Miniature Photometric Vision-Based Tactile Sensor

A Stable and Efficient Vision-Based Tactile Sensor with Tactile Detection Using Neural Network.