Abstract:This work proposes a novel multimodal tactile sensor utilizing triboelectric mechanism to simultaneously decouple three modalities: materials, curvatures, and pressure. Integrated with deep learning, an object recognition system can be achieved with high accuracy. It also can be further developed to detect object softness under different touch gestures, highlighting its potential for real‐world applications in robotics. Recent developments in robotics increasingly highlight the importance of sensing technology, especially tactile perception, in enabling robots to effectively engage with their environment and interpret physical interactions. Due to power efficiency and low cost, the triboelectric mechanism has been frequently studied for measuring pressure and identifying materials to enhance robot perception. Nevertheless, there has been limited exploration of using the triboelectric effect to detect curved surfaces, despite their prevalence in daily lives. Here, a triboelectric multimodal tactile sensor (TMTS) of multilayered structural design is proposed to recognize distinct materials, curvatures, and pressure simultaneously, thus decoupling different modalities to enable more accurate detection. By attaching sensors to robotic fingertips and leveraging deep learning analytics, the quantitative curvature measurement provides more precise insights into an object's detailed geometric characteristics rather than merely assessing its overall shape, hence achieving automatic recognition of 12 grasped objects with 99.2% accuracy. The sensor can be further used to accurately recognize the softness of objects under different touch gestures of a robotic hand, achieving a 94.1% accuracy, demonstrating its significant potential for wide‐ranging applications in a future robotic‐enabled intelligent society.

A NOVEL VISION-BASED TACTILE SENSOR USING PARTICLE IMAGE VELOCIMETRY FOR MULTI-MODAL OBJECT DETECTION AND FORCE SENSING

Robot Tactile Sensing: Vision Based Tactile Sensor for Force Perception

Fingerprint-inspired Dual-mode Pressure Sensor for Robotic Static and Dynamic Perception

Multidimensional Tactile Sensor with a Thin Compound Eye-Inspired Imaging System

A novel tactile sensor with multimodal vision and tactile units for multifunctional robot interaction

Development of Flexible Tactile Sensor for the Envelop of Curved Robotic Hand Finger in Grasping Force Sensing

A High-Resolution, Ultrabroad-Range and Sensitive Capacitive Tactile Sensor Based on a CNT/PDMS Composite for Robotic Hands.

A Dual-Modal Vision-Based Tactile Sensor for Robotic Hand Grasping.

A soft thumb-sized vision-based sensor with accurate all-round force perception

DotView: A Low-Cost Compact Tactile Sensor for Pressure, Shear, and Torsion Estimation

Visual Tactile Sensor Based Force Estimation for Position-Force Teleoperation

Towards vision-based robotic skins: a data-driven, multi-camera tactile sensor

A Vision-Based Tactile Sensing System for Multimodal Contact Information Perception via Neural Network

A Miniaturised Camera-based Multi-Modal Tactile Sensor

Design, Optimization and Application of Tactile Sensor Based on Virtual Binocular Vision

A Multifunctional Tactile Sensory System for Robotic Intelligent Identification and Manipulation Perception

9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation

Vision-Based Tactile Sensor Mechanism for the Estimation of Contact Position and Force Distribution Using Deep Learning

Augmented Tactile Perception of Robotic Fingers Enabled by AI‐Enhanced Triboelectric Multimodal Sensors

F-TOUCH Sensor: Concurrent Geometry Perception and Multi-Axis Force Measurement

Design a Multifunctional Soft Tactile Sensor Enhanced by Machine Learning Approaches