Abstract:Electrical resistance tomography (ERT) can be used to create large-scale soft tactile sensors that are flexible and robust. Good performance requires a fast and accurate mapping from the sensor’s sequential voltage measurements to the distribution of force across its surface. However, particularly with multiple contacts, this task is challenging for both previously developed approaches: physics-based modeling and end-to-end data-driven learning. Some promising results were recently achieved using sim-to-real transfer learning, but estimating multiple contact locations and accurate contact forces remains difficult because simulations tend to be less accurate with a high number of contact locations and/or high force. This paper introduces a modular hybrid method that combines simulation data synthesized from an electromechanical finite element model with real measurements collected from a new ERT-based tactile sensor. We use about 290 000 simulated and 90000 real measurements to train two deep neural networks: the first (Transfer-Net) captures the inevitable gap between simulation and reality, and the second (Recon-Net) reconstructs contact forces from voltage measurements. The number of contacts, contact locations, force magnitudes, and contact diameters are evaluated for a manually collected multi-contact dataset of 150 measurements. Our modular pipeline’s results outperform predictions by both a physics-based model and end-to-end learning. Note to Practitioners–ERT-based tactile sensors use high-speed voltage measurements from electrodes distributed over a piezoresistive area to output a force map that shows where contact is occurring, and how strong each contact is. Such sensors hold promise for giving robots and other devices a sense of touch over large surfaces with low hardware complexity. However, the software problem of converting voltages to an accurate force map has not previously been solved well, requiring either extensive model calibration or extensive contact data collection. This paper suggests a hybrid approach where a straightforward physics model simulates multi-contact experiments that are too costly to acquire in reality and a practical automatic indentation setup acquires real but geometrically limited multi-contact data. Although the number of real measurements required to learn the discrepancy between the sensor and the model is still large due to the inherent inverse nature of ERT-based tactile sensors, our combination of simulation and deep networks achieves better performance than either physical modeling or learning alone. This approach can advance practical large-area tactile sensing for industrial automation systems where multiple contacts occur, such as in manufacturing and assistive robotics. It could also likely be adapted to other nonlinear inverse problems.

Rectified Artificial Neural Networks for Long-term Force Sensing in Piezoelectric Touch Panels

Fingerprint-inspired Dual-mode Pressure Sensor for Robotic Static and Dynamic Perception

Machine learning-coupled tactile recognition with high spatiotemporal resolution based on cross-striped nanocarbon piezoresistive sensor array

Development of a Novel Strategy Based on In-Process Compensation of Charge Leakage for Static Force Measurement by Piezoelectric Force Sensors

Deep‐Learning‐Based Prediction of Long‐Term Piezoresistive Sensing Performance of MXene/Aramid Nanofiber Sensors

3D force identification and prediction using deep learning based on a Gelsight-structured sensor

Three-dimensional Force Detection and Decoupling of a Flexible Tactile Sensor Array based on Porous Composite Piezoresistive Materials

Artificial Tactile Sensory Finger for Contact Pattern Identification Based on High Spatiotemporal Piezoresistive Sensor Array

Actuation Force Modeling and Identification for Piezoelectric Actuator with Neural Network Compensator and Nonlinear Hysteresis

Ground Force Precision Calibration Method for Customized Piezoresistance Sensing Flexible Force Measurement Mat

Multimodal Surface Sensing Based on Hybrid Flexible Triboelectric and Piezoresistive Sensor.

A Learning-Based Multisensor Fingertip Force Measurement Method With Fast Data Generation

Kinetic and Kinematic Sensors-free Approach for Estimation of Continuous Force and Gesture in sEMG Prosthetic Hands

Design and Analysis of a Piezoelectric Material Based Touch Screen with Additional Pressure and Its Acceleration Measurement Functions

Finite element modeling of the electrical impedance tomography technique driven by machine learning

Recent Advances in Resistive Sensor Technology for Tactile Perception: A Review

Predicting the Force Map of an ERT-Based Tactile Sensor Using Simulation and Deep Networks

Self-Powered Force Sensor Based on Thinned Bulk PZT for Real-Time Cutaneous Activities Monitoring

Perception of Static and Dynamic Forces with a Bio-inspired Tactile Fingertip

Semg-Upper Limb Interaction Force Estimation Framework Based on Residual Network and Bidirectional Long Short-Term Memory Network

A Visual-Tactile Coulping Mechanism Sensor for Real-Time Force Calibration