Abstract:Electrical resistance tomography (ERT) can be used to create large-scale soft tactile sensors that are flexible and robust. Good performance requires a fast and accurate mapping from the sensor’s sequential voltage measurements to the distribution of force across its surface. However, particularly with multiple contacts, this task is challenging for both previously developed approaches: physics-based modeling and end-to-end data-driven learning. Some promising results were recently achieved using sim-to-real transfer learning, but estimating multiple contact locations and accurate contact forces remains difficult because simulations tend to be less accurate with a high number of contact locations and/or high force. This paper introduces a modular hybrid method that combines simulation data synthesized from an electromechanical finite element model with real measurements collected from a new ERT-based tactile sensor. We use about 290 000 simulated and 90000 real measurements to train two deep neural networks: the first (Transfer-Net) captures the inevitable gap between simulation and reality, and the second (Recon-Net) reconstructs contact forces from voltage measurements. The number of contacts, contact locations, force magnitudes, and contact diameters are evaluated for a manually collected multi-contact dataset of 150 measurements. Our modular pipeline’s results outperform predictions by both a physics-based model and end-to-end learning. Note to Practitioners–ERT-based tactile sensors use high-speed voltage measurements from electrodes distributed over a piezoresistive area to output a force map that shows where contact is occurring, and how strong each contact is. Such sensors hold promise for giving robots and other devices a sense of touch over large surfaces with low hardware complexity. However, the software problem of converting voltages to an accurate force map has not previously been solved well, requiring either extensive model calibration or extensive contact data collection. This paper suggests a hybrid approach where a straightforward physics model simulates multi-contact experiments that are too costly to acquire in reality and a practical automatic indentation setup acquires real but geometrically limited multi-contact data. Although the number of real measurements required to learn the discrepancy between the sensor and the model is still large due to the inherent inverse nature of ERT-based tactile sensors, our combination of simulation and deep networks achieves better performance than either physical modeling or learning alone. This approach can advance practical large-area tactile sensing for industrial automation systems where multiple contacts occur, such as in manufacturing and assistive robotics. It could also likely be adapted to other nonlinear inverse problems.

Adaptive Optimal Electrical Resistance Tomography for Large-Area Tactile Sensing.

Data-driven Electrical Resistance Tomography for Robotic Large-Area Tactile Sensing

A Large-Area Flexible Tactile Sensor for Multi-Touch and Force Detection Using Electrical Impedance Tomography

Strategy for Optimization of Arbitrary Electrode Distribution for 3D Electrical Resistivity Tomography in Urban Zones

The Selection Strategy of Optimized Arrays for 3-D Electrical Resistivity Tomography

A High-Resolution, Ultrabroad-Range and Sensitive Capacitive Tactile Sensor Based on a CNT/PDMS Composite for Robotic Hands.

Data-efficient Tactile Sensing with Electrical Impedance Tomography

A Large-area Tactile Sensor for Distributed Force Sensing Using Highly Sensitive Piezoresistive Sponge

Large-Scale Integrated Flexible Tactile Sensor Array for Sensitive Smart Robotic Touch

Flexible Tactile Sensor Array for Distributed Tactile Sensing and Slip Detection in Robotic Hand Grasping

Progress on Flexible Tactile Sensors in Robotic Applications on Objects Properties Recognition, Manipulation and Human-Machine Interactions

Electrical Impedance Tomography-Based Electronic Skin for Multi-Touch Tactile Sensing Using Hydrogel Material and FISTA Algorithm

Large‐Area Hand‐Covering Elastomeric Electronic Skin Sensor with Distributed Multifunctional Sensing Capability

A Novel Inverse Solution Of Contact Force Based On A Sparse Tactile Sensor Array

An ERT-based Robotic Skin with Sparsely Distributed Electrodes: Structure, Fabrication, and DNN-based Signal Processing

Obstacle Avoidance Path Planning for Robotic Arm Based on EIT Tactile Sensing

3-D-Curved Iontronic Tactile Sensor and Denoising Method for Physical Human–Robot Interactions

Predicting the Force Map of an ERT-Based Tactile Sensor Using Simulation and Deep Networks

Triboelectrification-enabled Touch Sensing for Self-Powered Position Mapping and Dynamic Tracking by a Flexible and Area-Scalable Sensor Array

Wireless and Flexible Tactile Sensing Array Based on an Adjustable Resonator with Machine‐Learning Perception

Touch Sensors with Overlapping Signals: Concept Investigation on Planar Sensors with Resistive or Optical Transduction