Abstract:Data glove-based dynamic gestures contain rich human motion intentions, which are reliant on the hand–body information that comes from multiindividual sensors attached. However, present gesture recognition with such wearable sensor devices tends to depend heavily on handcrafted features and ignore the critical channel and interfeature information. To address this problem, a novel convolutional-transformer-based recognition architecture termed the spatial–temporal feature-attention transformer network (STFTnet) is proposed in this study. Specifically, the acquired data from multiple sensors of the data glove are sequentially processed with a spatial–temporal sensor features embedding branch, a transformer encoder block, and the final gesture classifier. A multisensor feature attention (MFA) block and an improved depth-separable convolution block of the first branch are developed to effectively extract low-level spatial and local temporal features, while the multihead self-attention-based transformer block further concentrates on capturing the global context information. The gesture classifier is used to achieve the final classification successfully. To evaluate the efficacy of the proposed approach, extensive experiments are conducted on two publicly available datasets of pelvic closed reduction action dataset and the UC2017 Hand Gesture (HG) Dataset, and one self-built gesture control command dataset. Compared to the other state-of-the-art deep learning (DL)-based algorithms, an average accuracy of 95.75%, 100%, 99.72% and recognition time of 10.71, 11.92, and 11.24 ms has been achieved. These results indicate that the proposed network effectively enhances the recognition performance of the dynamic gesture of data gloves, while fulfilling requirements of the further real-time application.

Trajectory Image Based Dynamic Gesture Recognition with Convolutional Neural Networks

Dynamic hand gesture recognition using hidden Markov models

Research on Dynamic Gesture Recognition with Low-Pixel ToF-Sensors

Dynamic gesture recognition based on 2D convolutional neural network and feature fusion

A Comprehensive Evaluation of Moving Static Gesture Recognition with Convolutional Networks

Dynamic gesture tracking and recognition algorithm based on deep learning

Dynamic Gesture Recognition Algorithm Based on 3D Convolutional Neural Network

3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks

Visual Hand Gesture Recognition With Convolution Neural Network

Gesture recognition based on deep deformable 3D convolutional neural networks

Gesture Feature Extraction and Recognition Based on Image Processing

Dynamic Gesture Recognition Based on Two-Scale 3-D-ConvNeXt

WristCam: A Wearable Sensor for Hand Trajectory Gesture Recognition and Intelligent Human–Robot Interaction

A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves

Short-Term Temporal Convolutional Networks for Dynamic Hand Gesture Recognition

Gesture Recognition Algorithm of Human Motion Target Based on Deep Neural Network

Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks

Gesture Tracking and Recognition Algorithm for Dynamic Human Motion Using Multimodal Deep Learning

EfficientNetV2-based dynamic gesture recognition using transformed scalogram from triaxial acceleration signal

Application of Gesture Recognition Based on Spatiotemporal Graph Convolution Network in Virtual Reality Interaction

Improved mKLT and low layered HG-CNN based dynamic gesture recognition hardware system