Abstract:With an increasing number of people on the planet today, innovative human-computer interaction technologies and approaches may be employed to assist individuals in leading more fulfilling lives. Gesture-based technology has the potential to improve the safety and well-being of impaired people, as well as the general population. Recognizing gestures from video streams is a difficult problem because of the large degree of variation in the characteristics of each motion across individuals. In this article, we propose applying deep learning methods to recognize automated hand gestures using RGB and depth data. To train neural networks to detect hand gestures, any of these forms of data may be utilized. Gesture-based interfaces are more natural, intuitive, and straightforward. Earlier study attempted to characterize hand motions in a number of contexts. Our technique is evaluated using a vision-based gesture recognition system. In our suggested technique, image collection starts with RGB video and depth information captured with the Kinect sensor and is followed by tracking the hand using a single shot detector Convolutional Neural Network (SSD-CNN). When the kernel is applied, it creates an output value at each of the m n locations. Using a collection of convolutional filters, each new feature layer generates a defined set of gesture detection predictions. After that, we perform deep dilation to make the gesture in the image masks more visible. Finally, hand gestures have been detected using the well-known classification technique SVM. Using deep learning we recognize hand gestures with higher accuracy of 93.68% in RGB passage, 83.45% in the depth passage, and 90.61% in RGB-D conjunction on the SKIG dataset compared to the state-of-the-art. In the context of our own created Different Camera Orientation Gesture (DCOG) dataset we got higher accuracy of 92.78% in RGB passage, 79.55% in the depth passage, and 88.56% in RGB-D conjunction for the gestures collected in 0-degree angle. Moreover, the framework intends to use unique methodologies to construct a superior vision-based hand gesture recognition system.

Fingertip in the Eye: A cascaded CNN pipeline for the real-time fingertip detection in egocentric videos

Deep Learning Based Hand Gesture Recognition in Complex Scenes

A real-time hand gesture recognition method

Two-stream Convolutional Neural Network for Accurate RGB-D Fingertip Detection Using Depth and Edge Information.

End-to-End Spatial Transform Face Detection and Recognition

Accurate Fingertip Detection from Binocular Mask Images.

Deep Multi-Task Learning based Fingertip Detection

Real-time Fingertip Tracking and Gesture Recognition Using RGB-D Camera

Unified learning approach for egocentric hand gesture recognition and fingertip detection

Hand Segmentation and Fingertip Tracking from Depth Camera Images Using Deep Convolutional Neural Network and Multi-task SegNet

Faster R-CNN with Inception V2 for Fingertip Detection in Homogenous Background Image

Attention Based Dual Branches Fingertip Detection Network and Virtual Key System

Two-Stream Binocular Network: Accurate Near Field Finger Detection Based On Binocular Images

Single Shot Detector CNN and Deep Dilated Masks for Vision-Based Hand Gesture Recognition From Video Sequences

A Lightweight Hand-Gesture Recognition Network With Feature Fusion Prefiltering and FMCW Radar Spatial Angle Estimation

Fingertip Detection and Tracking Using 2D and 3D Information

Cascaded hierarchical CNN for 2D hand pose estimation from a single color image

SqueezeNet and Fusion Network-Based Accurate Fast Fully Convolutional Network for Hand Detection and Gesture Recognition

Detecting Hands in Egocentric Videos: Towards Action Recognition

Real-Time Hand Gesture Spotting and Recognition Using RGB-D Camera and 3D Convolutional Neural Network

Hand Gestures Recognition in Videos Taken with Lensless Camera