Abstract:Since tactile sensing provides rich and delicate sensations, touch-based object recognition has attracted public attention and has been extensively developed for robots. However, robotic grasping recognition in real-life scenarios is highly challenging due to the complexity of real-life objects in shapes, sizes, and other details, as well as the uncertainty of real grabs in orientations and locations. Here, we propose a novel robotic tactile sensing method, utilizing the spatiotemporal sensing of multimodal tactile sensors acquired during hand grasping to simultaneously perceive multi-attributes of the grasped object, including thermal conductivity, thermal diffusivity, surface roughness, contact pressure, and temperature. Multimodal perception of thermal attributes (thermal conductivity, diffusivity, and temperature) and mechanical attributes (roughness and contact pressure) greatly enhance the robotic ability to recognize objects. To further overcome the complexity and uncertainty in real-life grasping recognition, inspired by human logical reasoning “from easy to hard” in solving puzzles, we propose a novel cascade classifier using multilayered long short-term memory neural networks to hierarchically identify objects according to their features. With the enhanced multimodal perception ability of tactile sensors and the novel cascade classifier, the robotic grasping recognition achieves a high recognition accuracy of 98.85% in discriminating diverse garbage objects, showing excellent generalizability. The proposed spatiotemporal tactile sensing with logical reasoning strategy overcomes the difficulty of robotic object recognition in complex real-life scenes and facilitates its practical applications in our daily lives.

A Multimodal Robust Recognition Method for Grasping Objects with Robot Flexible Grippers

A soft robotic system imitating the multimodal sensory mechanism of human fingers for intelligent grasping and recognition

A Robotic Semantic Grasping Method for Pick-and-place Tasks

Intelligent Soft Robotic Gripper Enabled by Multimodal Sensors and Deep Learning.

Robotic Grasping Recognition Using Multi-Modal Deep Extreme Learning Machine

A robot grasping detection network based on flexible selection of multi-modal feature fusion structure

Enhanced robotic tactile perception with spatiotemporal sensing and logical reasoning for robust object recognition

Design of a Tactile Sensing Robotic Gripper and Its Grasping Method

Robotic Grasp Detection Using Extreme Learning Machine.

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Multifingered Grasping Based on Multimodal Reinforcement Learning

A Vision-based Robot Grasping System

Magnetostrictive-based multimodal tactile sensors for object recognition

Predict Robot Grasp Outcomes Based On Multi-Modal Information

A multisensory-feedback tactile glove with dense coverage of sensing arrays for object recognition

EfficientGrasp: A Unified Data-Efficient Learning to Grasp Method for Multi-Fingered Robot Hands

Hybrid Robotic Grasping with a Soft Multimodal Gripper and a Deep Multistage Learning Scheme

Robotic Grasp Detection Network Based on Improved Deformable Convolution and Spatial Feature Center Mechanism

LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion

sEMG-Based Robust Recognition of Grasping Postures with a Machine Learning Approach for Low-Cost Hand Control

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection