Abstract:The skeleton based gesture recognition is gaining more popularity due to its wide possible applications. The key issues are how to extract discriminative features and how to design the classification model. In this paper, we first leverage a robust feature descriptor, path signature (PS), and propose three PS features to explicitly represent the spatial and temporal motion characteristics, i.e., spatial PS (S PS), temporal PS (T PS) and temporal spatial PS (T S PS). Considering the significance of fine hand movements in the gesture, we propose an ”attention on hand” (AOH) principle to define joint pairs for the S PS and select single joint for the T PS. In addition, the dyadic method is employed to extract the T PS and T S PS features that encode global and local temporal dynamics in the motion. Secondly, without the recurrent strategy, the classification model still faces challenges on temporal variation among different sequences. We propose a new temporal transformer module (TTM) that can match the sequence key frames by learning the temporal shifting parameter for each input. This is a learning-based module that can be included into standard neural network architecture. Finally, we design a multi-stream fully connected layer based network to treat spatial and temporal features separately and fused them together for the final result. We have tested our method on three benchmark gesture datasets, i.e., ChaLearn 2016, ChaLearn 2013 and MSRC-12. Experimental results demonstrate that we achieve the state-of-the-art performance on skeleton-based gesture recognition with high computational efficiency.

Motion-Pose Recurrent Neural Network With Instantaneous Kinematic Descriptor For Skeleton Based Gesture Detection And Recognition

Motion Feature Augmented Recurrent Neural Network for Skeleton-based Dynamic Hand Gesture Recognition

Temporal-Spatial Deformable Pose Network for Skeleton-Based Gesture Recognition

CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

Skeleton-Based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module.

Gesture Recognition Method Based On Deep Learning

MFA-Net: Motion Feature Augmented Network for Dynamic Hand Gesture Recognition from Skeletal Data

Gesture Recognition Algorithm of Human Motion Target Based on Deep Neural Network

A Skeleton Object Detection-Based Dynamic Gesture Recognition Method

Recognizing Skeleton-Based Hand Gestures by a Spatio-Temporal Network

Skeleton Based Dynamic Hand Gesture Recognition using Short Term Sampling Neural Networks (STSNN).

A Local Spatial–Temporal Synchronous Network to Dynamic Gesture Recognition

Bidirectional Independently Recurrent Neural Network for Skeleton-based Hand Gesture Recognition

Gesture Recognition Matching Based on Dynamic Skeleton

Learning Local Part Motion Representation for Skeleton-based Action Recognition

Human Skeleton Tree Recurrent Neural Network with Joint Relative Motion Feature for Skeleton Based Action Recognition

Gesture Recognition Using Enhanced Depth Motion Map and Static Pose Map

Human Body Gesture Recognition Method Based on Deep Learning

Dynamic long short-term memory network for skeleton-based gait recognition

Continuous Motion Recognition Using Multiple Time Constant Recurrent Neural Network With A Deep Network Model

Skeleton-Based Dynamic Hand Gesture Recognition Using an Enhanced Network with One-Shot Learning