Abstract:The skeleton based gesture recognition is gaining more popularity due to its wide possible applications. The key issues are how to extract discriminative features and how to design the classification model. In this paper, we first leverage a robust feature descriptor, path signature (PS), and propose three PS features to explicitly represent the spatial and temporal motion characteristics, i.e., spatial PS (S PS), temporal PS (T PS) and temporal spatial PS (T S PS). Considering the significance of fine hand movements in the gesture, we propose an ”attention on hand” (AOH) principle to define joint pairs for the S PS and select single joint for the T PS. In addition, the dyadic method is employed to extract the T PS and T S PS features that encode global and local temporal dynamics in the motion. Secondly, without the recurrent strategy, the classification model still faces challenges on temporal variation among different sequences. We propose a new temporal transformer module (TTM) that can match the sequence key frames by learning the temporal shifting parameter for each input. This is a learning-based module that can be included into standard neural network architecture. Finally, we design a multi-stream fully connected layer based network to treat spatial and temporal features separately and fused them together for the final result. We have tested our method on three benchmark gesture datasets, i.e., ChaLearn 2016, ChaLearn 2013 and MSRC-12. Experimental results demonstrate that we achieve the state-of-the-art performance on skeleton-based gesture recognition with high computational efficiency.

Dynamic Hand Gesture Recognition Based On 3D Skeleton

Dynamic hand gesture recognition using hidden Markov models

Skeleton-based Dynamic Hand Gesture Recognition using 3D Depth Data

Dynamic Hand Gesture Recognition Based on Depth Information

Dynamic Gesture Recognition Research Using Kinect's Skeleton Information

Gesture Recognition Matching Based on Dynamic Skeleton

DHGD: Dynamic Hand Gesture Dataset for Skeleton-Based Gesture Recognition and Baseline Evaluations

Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human-Robot Interaction

Automatic 3D Skeleton-based Dynamic Hand Gesture Recognition Using Multi-Layer Convolutional LSTM.

Skeleton Based Dynamic Hand Gesture Recognition using LSTM and CNN

A Skeleton Object Detection-Based Dynamic Gesture Recognition Method

Dynamic Hand Gesture Recognition Using Motion Pattern and Shape Descriptors

Dynamic hand gesture early recognition based on Hidden Semi-Markov Models

Motion Feature Augmented Recurrent Neural Network for Skeleton-based Dynamic Hand Gesture Recognition

Hidden States Exploration for 3D Skeleton-Based Gesture Recognition.

Deep Learning for Hand Gesture Recognition on Skeletal Data

Skeleton-Based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module.

3D Skeletal Gesture Recognition Via Hidden States Exploration

Dynamic Hand Gesture Recognition Based on Two-channel Hybrid 3D-2D RBM

Dynamic Gesture Recognition Based On Improved Dtw Algorithm

Real-time 3D bare-hand gesture recognition using binocular vision videos