Abstract:Grasp classification is pivotal for understanding human interactions with objects, with wide-ranging applications in robotics, prosthetics, and rehabilitation. This study introduces a novel methodology utilizing a multisensory data glove to capture intricate grasp dynamics, including finger posture bending angles and fingertip forces. Our dataset comprises data collected from 10 participants engaging in grasp trials with 24 objects using the YCB object set. We evaluate classification performance under three scenarios: utilizing grasp posture alone, utilizing grasp force alone, and combining both modalities. We propose Glove-Net, a hybrid CNN-BiLSTM architecture for classifying grasp patterns within our dataset, aiming to harness the unique advantages offered by both CNNs and BiLSTM networks. This model seamlessly integrates CNNs' spatial feature extraction capabilities with the temporal sequence learning strengths inherent in BiLSTM networks, effectively addressing the intricate dependencies present within our grasping data. Our study includes findings from an extensive ablation study aimed at optimizing model configurations and hyperparameters. We quantify and compare the classification accuracy across these scenarios: CNN achieved 88.09%, 69.38%, and 93.51% testing accuracies for posture-only, force-only, and combined data, respectively. LSTM exhibited accuracies of 86.02%, 70.52%, and 92.19% for the same scenarios. Notably, the hybrid CNN-BiLSTM proposed model demonstrated superior performance with accuracies of 90.83%, 73.12%, and 98.75% across the respective scenarios. Through rigorous numerical experimentation, our results underscore the significance of multimodal grasp classification and highlight the efficacy of the proposed hybrid Glove-Net architectures in leveraging multisensory data for precise grasp recognition. These insights advance understanding of human-machine interaction and hold promise for diverse real-world applications.

Grasp Classification With Weft Knit Data Glove Using a Convolutional Neural Network

Glove-Net: Enhancing Grasp Classification with Multisensory Data and Deep Learning Approach

Gesture Classification in Electromyography Signals for Real-Time Prosthetic Hand Control Using a Convolutional Neural Network-Enhanced Channel Attention Model

Vision-aided grasp classification: design and evaluation of compact CNN for prosthetic hands

Variable Stiffness Object Recognition with a CNN-Bayes Classifier on a Soft Gripper

Grasp Stability Prediction Using Tactile Information.

Grasp Type Understanding — Classification, Localization and Clustering

Hand Grasps Recognition Using Single-Channel Transient-State Surface Electromyography

Quantification of finger grasps during activities of daily life using convolutional neural networks: A pilot study

Utilizing Gramian Angular Fields and Convolution Neural Networks in Flex Sensors Glove for Human–Computer Interaction

Comparing classification techniques for identification of grasped objects

Machine Learning-Based Gesture Recognition Glove: Design and Implementation

A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves

Deep Learning Method for Grasping Novel Objects Using Dexterous Hands

Heterogeneous Hand Guise Classification Based on Surface Electromyographic Signals Using Multichannel Convolutional Neural Network

Capturing complex hand movements and object interactions using machine learning-powered stretchable smart textile gloves

Multi-Fingered In-Hand Manipulation with Various Object Properties Using Graph Convolutional Networks and Distributed Tactile Sensors

Hand Gesture Classification through Time Frequency Images with One-to-One, All-to-One, and Cross-Validation Approaches.

Gesture Recognition Based on a Convolutional Neural Network–Bidirectional Long Short-Term Memory Network for a Wearable Wrist Sensor with Multi-Walled Carbon Nanotube/Cotton Fabric Material

Innovative healthcare solutions: robust hand gesture recognition of daily life routines using 1D CNN