Abstract:Electrical resistance tomography (ERT) can be used to create large-scale soft tactile sensors that are flexible and robust. Good performance requires a fast and accurate mapping from the sensor’s sequential voltage measurements to the distribution of force across its surface. However, particularly with multiple contacts, this task is challenging for both previously developed approaches: physics-based modeling and end-to-end data-driven learning. Some promising results were recently achieved using sim-to-real transfer learning, but estimating multiple contact locations and accurate contact forces remains difficult because simulations tend to be less accurate with a high number of contact locations and/or high force. This paper introduces a modular hybrid method that combines simulation data synthesized from an electromechanical finite element model with real measurements collected from a new ERT-based tactile sensor. We use about 290 000 simulated and 90000 real measurements to train two deep neural networks: the first (Transfer-Net) captures the inevitable gap between simulation and reality, and the second (Recon-Net) reconstructs contact forces from voltage measurements. The number of contacts, contact locations, force magnitudes, and contact diameters are evaluated for a manually collected multi-contact dataset of 150 measurements. Our modular pipeline’s results outperform predictions by both a physics-based model and end-to-end learning. Note to Practitioners–ERT-based tactile sensors use high-speed voltage measurements from electrodes distributed over a piezoresistive area to output a force map that shows where contact is occurring, and how strong each contact is. Such sensors hold promise for giving robots and other devices a sense of touch over large surfaces with low hardware complexity. However, the software problem of converting voltages to an accurate force map has not previously been solved well, requiring either extensive model calibration or extensive contact data collection. This paper suggests a hybrid approach where a straightforward physics model simulates multi-contact experiments that are too costly to acquire in reality and a practical automatic indentation setup acquires real but geometrically limited multi-contact data. Although the number of real measurements required to learn the discrepancy between the sensor and the model is still large due to the inherent inverse nature of ERT-based tactile sensors, our combination of simulation and deep networks achieves better performance than either physical modeling or learning alone. This approach can advance practical large-area tactile sensing for industrial automation systems where multiple contacts occur, such as in manufacturing and assistive robotics. It could also likely be adapted to other nonlinear inverse problems.

An ERT-based Robotic Skin with Sparsely Distributed Electrodes: Structure, Fabrication, and DNN-based Signal Processing

Design of an EIT-based Flexible Tactile Sensor with Center Electrodes

A multifunctional electronic skin based on patterned metal films for tactile sensing with a broad linear response range

Data-driven Electrical Resistance Tomography for Robotic Large-Area Tactile Sensing

Fabrication of a Tactile Sensor for Artificial Skin Based on Electrical Impedance Tomography

Implementation and Evaluation of a Soft Tactile Sensor Using Electrical Impedance Tomography

A Large-Area Flexible Tactile Sensor for Multi-Touch and Force Detection Using Electrical Impedance Tomography

A biomimetic elastomeric robot skin using electrical impedance and acoustic tomography for tactile sensing

A Deformable Smart Skin for Continuous Sensing Based on Electrical Impedance Tomography

Stretchable, Flexible, Scalable Smart Skin Sensors for Robotic Position and Force Estimation

Large-Area High-Performance Flexible Pressure Sensor with Carbon Nanotube Active Matrix for Electronic Skin

Adaptive Optimal Electrical Resistance Tomography for Large-Area Tactile Sensing.

A Piezoresistive Tactile Sensor for a Large Area Employing Neural Network.

Self‐Powered and Interface‐Independent Tactile Sensors Based on Bilayer Single‐Electrode Triboelectric Nanogenerators for Robotic Electronic Skin

Extremely durable electrical impedance tomography–based soft and ultrathin wearable e-skin for three-dimensional tactile interfaces

Artificial Sensitive Skin for Robotics Based on Electrical Impedance Tomography

Tattoo-like Epidermal Electronics As Skin Sensors for Human Machine Interfaces

Bioinspired electronic-skin for proximity and pressure detection in robot active sensing

Triboelectric and Iontronic Dual-Responsive Bioinspired Ionic Skin for Human–like Dexterous Robotic Manipulation

Predicting the Force Map of an ERT-Based Tactile Sensor Using Simulation and Deep Networks

Flexible Electronic Skin Sensor Based on Regenerated Cellulose/carbon Nanotube Composite Films