Abstract:Touch is a crucial sensing modality that provides rich information about object properties and interactions with the physical environment. Humans and robots both benefit from using touch to perceive and interact with the surrounding environment (Johansson and Flanagan, 2009; Li et al., 2020; Calandra et al., 2017). However, no existing systems provide rich, multi-modal digital touch-sensing capabilities through a hemispherical compliant embodiment. Here, we describe several conceptual and technological innovations to improve the digitization of touch. These advances are embodied in an artificial finger-shaped sensor with advanced sensing capabilities. Significantly, this fingertip contains high-resolution sensors (~8.3 million taxels) that respond to omnidirectional touch, capture multi-modal signals, and use on-device artificial intelligence to process the data in real time. Evaluations show that the artificial fingertip can resolve spatial features as small as 7 um, sense normal and shear forces with a resolution of 1.01 mN and 1.27 mN, respectively, perceive vibrations up to 10 kHz, sense heat, and even sense odor. Furthermore, it embeds an on-device AI neural network accelerator that acts as a peripheral nervous system on a robot and mimics the reflex arc found in humans. These results demonstrate the possibility of digitizing touch with superhuman performance. The implications are profound, and we anticipate potential applications in robotics (industrial, medical, agricultural, and consumer-level), virtual reality and telepresence, prosthetics, and e-commerce. Toward digitizing touch at scale, we open-source a modular platform to facilitate future research on the nature of touch.

Re-evaluating Parallel Finger-tip Tactile Sensing for Inferring Object Adjectives: An Empirical Study

DotTip: Enhancing Dexterous Robotic Manipulation with a Tactile Fingertip Featuring Curved Perceptual Morphology

Development of Flexible Tactile Sensor for the Envelop of Curved Robotic Hand Finger in Grasping Force Sensing

Progress on Flexible Tactile Sensors in Robotic Applications on Objects Properties Recognition, Manipulation and Human-Machine Interactions

Perception of Static and Dynamic Forces with a Bio-inspired Tactile Fingertip

Augmenting Off-the-Shelf Grippers with Tactile Sensing

Recent progress on tactile object recognition

A High-Resolution, Ultrabroad-Range and Sensitive Capacitive Tactile Sensor Based on a CNT/PDMS Composite for Robotic Hands.

GelTip: A Finger-shaped Optical Tactile Sensor for Robotic Manipulation

PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation

High Resolution Tactile Sensors for Curved Robotic Fingertips

Fingertip Contact Force Direction Control using Tactile Feedback

Digitizing Touch with an Artificial Multimodal Fingertip

A Dual-Modal Vision-Based Tactile Sensor for Robotic Hand Grasping.

Sensor Embedded Soft Fingertip for Precise Manipulation and Softness Recognition

Multifunctional Tactile Sensors for Object Recognition

GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger

Imagine2touch: Predictive Tactile Sensing for Robotic Manipulation using Efficient Low-Dimensional Signals

A Petal-Array Capacitive Tactile Sensor with Micro-Pin for Robotic Fingertip Sensing

FingerTac -- An Interchangeable and Wearable Tactile Sensor for the Fingertips of Human and Robot Hands

DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexterity