Abstract:The dynamic hand skeleton data have become increasingly attractive to widely studied for the recognition of hand gestures that contain 3D coordinates of hand joints. Many researchers have been working to develop skeleton-based hand gesture recognition systems using various discriminative spatial-temporal attention features by calculating the dependencies between the joints. However, these methods may face difficulties in achieving high performance and generalizability due to their inefficient features. To overcome these challenges, we proposed a Multi-branch attention-based graph and a general deep-learning model to recognize hand gestures by extracting all possible types of skeleton-based features. We used two graph-based neural network channels in our multi-branch architectures and one general neural network channel. In graph-based neural network channels, one channel first uses the spatial attention module and then the temporal attention module to produce the spatial-temporal features. In contrast, we produced temporal-spatial features in the second channel using the reverse sequence of the first branch. The last branch extracts general deep learning-based features using a general deep neural network module. The final feature vector was constructed by concatenating the spatial-temporal, temporal-spatial, and general features and feeding them into the fully connected layer. We included position embedding and mask operation for both spatial and temporal attention modules to track the node's sequence and reduce the system's computational complexity. Our model achieved 94.12%, 92.00%, and 97.01% accuracy after evaluation with MSRA, DHG, and SHREC'17 benchmark datasets, respectively. The high-performance accuracy and low computational cost proved that the proposed method outperformed the existing state-of-the-art methods.

Spatial-Temporal Attention Res-TCN for Skeleton-Based Dynamic Hand Gesture Recognition

Spatio-Temporal Dynamic Attention Graph Convolutional Network Based on Skeleton Gesture Recognition

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Selective spatiotemporal features learning for dynamic gesture recognition

Short-Term Temporal Convolutional Networks for Dynamic Hand Gesture Recognition

HAN: An Efficient Hierarchical Self-Attention Network for Skeleton-Based Gesture Recognition

An Efficient Graph Convolution Network for Skeleton-Based Dynamic Hand Gesture Recognition

MVHANet: Multi-view Hierarchical Aggregation Network for Skeleton-Based Hand Gesture Recognition

Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognition

Dynamic Gesture Recognition Method Based on Improved R(2+1)D

Spatial Temporal Graph Attention Network for Skeleton-Based Action Recognition

Temporal Decoupling Graph Convolutional Network for Skeleton-Based Gesture Recognition

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition

High Performance Gesture Recognition Via Effective and Efficient Temporal Modeling.

Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

STCN-GR: Spatial-Temporal Convolutional Networks for Surface-Electromyography-Based Gesture Recognition

Dynamic Hand Gesture Recognition Using Multi-Branch Attention Based Graph and General Deep Learning Model

Spatiotemporal features representation with dynamic mode decomposition for hand gesture recognition using deep neural networks

A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves

A Spatio-Temporal Graph Convolutional Network for Gesture Recognition from High-Density Electromyography

Learning Coupled Spatial-temporal Attention for Skeleton-based Action Recognition