A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves
Yingzhe Tang,Mingzhang Pan,Hongqi Li,Xinxin Cao
DOI: https://doi.org/10.1109/tim.2024.3400361
IF: 5.6
2024-05-24
IEEE Transactions on Instrumentation and Measurement
Abstract:Data glove-based dynamic gestures contain rich human motion intentions, which are reliant on the hand–body information that comes from multiindividual sensors attached. However, present gesture recognition with such wearable sensor devices tends to depend heavily on handcrafted features and ignore the critical channel and interfeature information. To address this problem, a novel convolutional-transformer-based recognition architecture termed the spatial–temporal feature-attention transformer network (STFTnet) is proposed in this study. Specifically, the acquired data from multiple sensors of the data glove are sequentially processed with a spatial–temporal sensor features embedding branch, a transformer encoder block, and the final gesture classifier. A multisensor feature attention (MFA) block and an improved depth-separable convolution block of the first branch are developed to effectively extract low-level spatial and local temporal features, while the multihead self-attention-based transformer block further concentrates on capturing the global context information. The gesture classifier is used to achieve the final classification successfully. To evaluate the efficacy of the proposed approach, extensive experiments are conducted on two publicly available datasets of pelvic closed reduction action dataset and the UC2017 Hand Gesture (HG) Dataset, and one self-built gesture control command dataset. Compared to the other state-of-the-art deep learning (DL)-based algorithms, an average accuracy of 95.75%, 100%, 99.72% and recognition time of 10.71, 11.92, and 11.24 ms has been achieved. These results indicate that the proposed network effectively enhances the recognition performance of the dynamic gesture of data gloves, while fulfilling requirements of the further real-time application.
engineering, electrical & electronic,instruments & instrumentation