Abstract:<p>Dynamic gesture recognition, which plays an essential role in human-computer interaction, has been widely investigated but not yet fully addressed. The challenge mainly lies in three folders: 1) to model both of the spatial appearance and the temporal evolution simultaneously; 2) to address the interference from the varied and complex background; 3) the requirement of real-time processing. In this paper, we address the above challenges by proposing a novel deep deformable 3D convolutional neural network for end-to-end learning, which not only gains impressive accuracy in challenging datasets but also can meet the requirement of the real-time processing. We propose three types of very deep 3D CNNs for gesture recognition, which can directly model the spatiotemporal information with their inherent hierarchical structure. To eliminate the background interference, a light-weight spatiotemporal deformable convolutional module is specially designed to augment the spatiotemporal sampling locations of the 3D convolution by learning additional offsets according to the preceding feature map. It can not only diversify the shape of the convolution kernel to better fit the appearance of the hands and arms, but also help the models pay more attention to the discriminative frames in the video sequence. The proposed method is evaluated on three challenging datasets, EgoGesture, Jester and Chalearn-IsoGD, and achieves the state-of-the-art performance on all of them. Our model ranked first on Jester's official leader-board until the submission time. The code and the trained models are released for better communication and future works<a class="workspace-trigger" href="#fn0001"><sup>1</sup></a>.</p>

3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks

Deep Learning Based Hand Gesture Recognition in Complex Scenes

Gesture recognition based on deep deformable 3D convolutional neural networks

Exploiting Recurrent Neural Networks and Leap Motion Controller for Sign Language and Semaphoric Gesture Recognition

Hand Gesture Recognition with Leap Motion

Short-Term Temporal Convolutional Networks for Dynamic Hand Gesture Recognition

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

A Comprehensive Evaluation of Moving Static Gesture Recognition with Convolutional Networks

A PointNet-Based Solution for 3D Hand Gesture Recognition

Hand Gesture Recognition for Game 3D Object Using The Leap Motion Controller with Backpropagation Method

Dynamic Gesture Recognition Algorithm Based on 3D Convolutional Neural Network

Visual Hand Gesture Recognition With Convolution Neural Network

Real-Time Hand Gesture Spotting and Recognition Using RGB-D Camera and 3D Convolutional Neural Network

Dynamic gesture recognition based on 2D convolutional neural network and feature fusion

Deep self-supervised learning with visualisation for automatic gesture recognition

CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human-Robot Interaction

Dynamic Gesture Recognition Based on Two-Scale 3-D-ConvNeXt

A Novel Autonomous Learning Framework to Enhance Semg-Based Hand Gesture Recognition Using Depth Information.

A Novel Hybrid Deep Learning Architecture for Dynamic Hand Gesture Recognition

3D Gesture Recognition Method Based on Faster R-CNN Network.