Abstract:Soft robots offer significant advantages in adaptability, safety, and dexterity compared to conventional rigid-body robots. However, it is challenging to equip soft robots with accurate proprioception and tactile sensing due to their high flexibility and elasticity. In this work, we describe the development of a vision-based proprioceptive and tactile sensor for soft robots called GelFlex, which is inspired by previous GelSight sensing techniques. More specifically, we develop a novel exoskeleton-covered soft finger with embedded cameras and deep learning methods that enable high-resolution proprioceptive sensing and rich tactile sensing. To do so, we design features along the axial direction of the finger, which enable high-resolution proprioceptive sensing, and incorporate a reflective ink coating on the surface of the finger to enable rich tactile sensing. We design a highly underactuated exoskeleton with a tendon-driven mechanism to actuate the finger. Finally, we assemble 2 of the fingers together to form a robotic gripper and successfully perform a bar stock classification task, which requires both shape and tactile information. We train neural networks for proprioception and shape (box versus cylinder) classification using data from the embedded sensors. The proprioception CNN had over 99% accuracy on our testing set (all six joint angles were within 1° of error) and had an average accumulative distance error of 0.77 mm during live testing, which is better than human finger proprioception. These proposed techniques offer soft robots the high-level ability to simultaneously perceive their proprioceptive state and peripheral environment, providing potential solutions for soft robots to solve everyday manipulation tasks. We believe the methods developed in this work can be widely applied to different designs and applications.

GelSight Svelte Hand: A Three-finger, Two-DoF, Tactile-rich, Low-cost Robot Hand for Dexterous Manipulation

GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing

GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger

EyeSight Hand: Design of a Fully-Actuated Dexterous Robot Hand with Integrated Vision-Based Tactile Sensors and Compliant Actuation

Exoskeleton-covered soft finger with vision-based proprioception and tactile sensing

GelLink: A Compact Multi-phalanx Finger with Vision-based Tactile Sensing and Proprioception

Elastic Tactile Sensor Glove for Dexterous Teaching by Demonstration

Sensorized Soft Skin for Dexterous Robotic Hands

Tactile SoftHand-A: 3D-Printed, Tactile, Highly-underactuated, Anthropomorphic Robot Hand with an Antagonistic Tendon Mechanism

A Passively Bendable, Compliant Tactile Palm with RObotic Modular Endoskeleton Optical (ROMEO) Fingers

GelSight360: An Omnidirectional Camera-Based Tactile Sensor for Dexterous Robotic Manipulation

GelSlim 4.0: Focusing on Touch and Reproducibility

GelTip Tactile Sensor for Dexterous Manipulation in Clutter

Tactile Model O: Fabrication and testing of a 3d-printed, three-fingered tactile robot hand

A Compliant, Underactuated Hand for Robust Manipulation

Design of a 3D-Printed Soft Robotic Hand With Integrated Distributed Tactile Sensing

A Robust Data-Driven Soft Sensory Glove for Human Hand Motions Identification and Replication

A 3D Printed Soft Robotic Hand With Embedded Soft Sensors for Direct Transition Between Hand Gestures and Improved Grasping Quality and Diversity

Dexterous object manipulation by a multi-fingered robotic hand with visual-tactile fingertip sensors

A Novel Design of Soft Robotic Hand with a Human-inspired Soft Palm for Dexterous Grasping

Design and Fabrication of a Soft Robotic Hand with Embedded Actuators and Sensors