Abstract:Grasp classification is pivotal for understanding human interactions with objects, with wide-ranging applications in robotics, prosthetics, and rehabilitation. This study introduces a novel methodology utilizing a multisensory data glove to capture intricate grasp dynamics, including finger posture bending angles and fingertip forces. Our dataset comprises data collected from 10 participants engaging in grasp trials with 24 objects using the YCB object set. We evaluate classification performance under three scenarios: utilizing grasp posture alone, utilizing grasp force alone, and combining both modalities. We propose Glove-Net, a hybrid CNN-BiLSTM architecture for classifying grasp patterns within our dataset, aiming to harness the unique advantages offered by both CNNs and BiLSTM networks. This model seamlessly integrates CNNs' spatial feature extraction capabilities with the temporal sequence learning strengths inherent in BiLSTM networks, effectively addressing the intricate dependencies present within our grasping data. Our study includes findings from an extensive ablation study aimed at optimizing model configurations and hyperparameters. We quantify and compare the classification accuracy across these scenarios: CNN achieved 88.09%, 69.38%, and 93.51% testing accuracies for posture-only, force-only, and combined data, respectively. LSTM exhibited accuracies of 86.02%, 70.52%, and 92.19% for the same scenarios. Notably, the hybrid CNN-BiLSTM proposed model demonstrated superior performance with accuracies of 90.83%, 73.12%, and 98.75% across the respective scenarios. Through rigorous numerical experimentation, our results underscore the significance of multimodal grasp classification and highlight the efficacy of the proposed hybrid Glove-Net architectures in leveraging multisensory data for precise grasp recognition. These insights advance understanding of human-machine interaction and hold promise for diverse real-world applications.

Tactile Grasp Stability Classification Based on Graph Convolutional Networks

Grasp Stability Prediction Using Tactile Information.

Robot Grasping Stability Prediction Network based on Feature-fusion and Feature-reconstruction of Tactile Information

Visual-Tactile Fusion for Robotic Stable Grasping

Grasp Stability Assessment Through Spatio-Temporal Attention Mechanism and Adaptive Gate Fusion

Grasp Stability Assessment Through the Fusion of Visual and Tactile Signals Using Deep Bilinear Network.

Grasp Stability Assessment Through Attention-Guided Cross-Modality Fusion and Transfer Learning

Efficient Grasp Detection Network with Gaussian-Based Grasp Representation for Robotic Manipulation

Tactile-GAT: tactile graph attention networks for robot tactile perception classification

Glove-Net: Enhancing Grasp Classification with Multisensory Data and Deep Learning Approach

Predict Robot Grasp Outcomes Based On Multi-Modal Information

Grasp State Assessment of Deformable Objects Using Visual-Tactile Fusion Perception

Multi-Fingered In-Hand Manipulation with Various Object Properties Using Graph Convolutional Networks and Distributed Tactile Sensors

Robotic tactile recognition and adaptive grasping control based on CNN-LSTM

Non-Matrix Tactile Sensors: How Can Be Exploited Their Local Connectivity For Predicting Grasp Stability?

Detecting Transitions from Stability to Instability in Robotic Grasping Based on Tactile Perception

Robotic grasping using visual and tactile sensing

Tactile-ViewGCN: Learning Shape Descriptor from Tactile Data using Graph Convolutional Network

CMG-Net: An End-to-End Contact-Based Multi-Finger Dexterous Grasping Network

Learning Optimal Grasping Posture of Multi-Fingered Dexterous Hands for Unknown Objects

Integrating High-Resolution Tactile Sensing into Grasp Stability Prediction