Abstract:Grasp classification is pivotal for understanding human interactions with objects, with wide-ranging applications in robotics, prosthetics, and rehabilitation. This study introduces a novel methodology utilizing a multisensory data glove to capture intricate grasp dynamics, including finger posture bending angles and fingertip forces. Our dataset comprises data collected from 10 participants engaging in grasp trials with 24 objects using the YCB object set. We evaluate classification performance under three scenarios: utilizing grasp posture alone, utilizing grasp force alone, and combining both modalities. We propose Glove-Net, a hybrid CNN-BiLSTM architecture for classifying grasp patterns within our dataset, aiming to harness the unique advantages offered by both CNNs and BiLSTM networks. This model seamlessly integrates CNNs' spatial feature extraction capabilities with the temporal sequence learning strengths inherent in BiLSTM networks, effectively addressing the intricate dependencies present within our grasping data. Our study includes findings from an extensive ablation study aimed at optimizing model configurations and hyperparameters. We quantify and compare the classification accuracy across these scenarios: CNN achieved 88.09%, 69.38%, and 93.51% testing accuracies for posture-only, force-only, and combined data, respectively. LSTM exhibited accuracies of 86.02%, 70.52%, and 92.19% for the same scenarios. Notably, the hybrid CNN-BiLSTM proposed model demonstrated superior performance with accuracies of 90.83%, 73.12%, and 98.75% across the respective scenarios. Through rigorous numerical experimentation, our results underscore the significance of multimodal grasp classification and highlight the efficacy of the proposed hybrid Glove-Net architectures in leveraging multisensory data for precise grasp recognition. These insights advance understanding of human-machine interaction and hold promise for diverse real-world applications.

Single-Grasp Deformable Object Discrimination: The Effect of Gripper Morphology, Sensing Modalities, and Action Parameters

Pregrasp Object Material Classification by a Novel Gripper Design with Integrated Spectroscopy

Inter-finger Small Object Manipulation With DenseTact Optical Tactile Sensor

Three Sliding Probes Placed on Forelimb Skin for Proprioceptive Feedback Differentially yet Complementarily Contribute to Hand Gesture Detection and Object-Size Discrimination

Simple Kinesthetic Haptics for Object Recognition

Enhancing Perception with Tactile Object Recognition in Adaptive Grippers for Human–Robot Interaction

Gaining a Sense of Touch. Physical Parameters Estimation using a Soft Gripper and Neural Networks

Design and Evaluation of Anthropomorphic Robotic Hand for Object Grasping and Shape Recognition

Learning thin deformable object manipulation with a multi-sensory integrated soft hand

Kinesthetic-based In-Hand Object Recognition with an Underactuated Robotic Hand

Get a Grip: Multi-Finger Grasp Evaluation at Scale Enables Robust Sim-to-Real Transfer

Multimodal Grasp Data Set: A Novel Visual–tactile Data Set for Robotic Manipulation

Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation

PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation

Grasping, Part Identification, and Pose Refinement in One Shot with a Tactile Gripper

Glove-Net: Enhancing Grasp Classification with Multisensory Data and Deep Learning Approach

A Dual-Modal Vision-Based Tactile Sensor for Robotic Hand Grasping.

Pose-free object classification from surface contact features in sequences of Robotic grasps

Learning to Detect Slip through Tactile Estimation of the Contact Force Field and its Entropy

Dynamic Modeling of Hand-Object Interactions via Tactile Sensing

DexiTac: Soft Dexterous Tactile Gripping