Abstract:Gesture and fingertip are becoming more and more important mediums for human-computer interaction (HCI). Therefore, algorithms of gesture recognition and fingertip detection have been extensively investigated. However, problems mainly remain in how to achieve a win-win situation between speed and accuracy, and how to deal with complex interaction environment. To rectify these problems, this paper proposes an attention-based dual branches network that can efficiently fulfill both fingertip detection and gesture recognition tasks. In order to deal with complex interaction environment, we combine both channel-wise attention and spatial-wise attention into the fingertip detection model. The extensive experiments demonstrate that our novel model is both effective and efficient. In the experiment, our proposed model achieves the average fingertip detection error at around 2.8 pixels in 640×480 video frame, and the average recognition accuracy among eight gestures reaches $99%$. Moreover, the average forward time is about 8 ms. Due to the light-weight design, this model can also achieve high-efficiency performance on CPU. In addition, we design a virtual key system based on our proposed model, which can allow users to complete the "clicking" operation naturally in virtual environment. Our proposed system can perform well with a single normal RGB camera without any pre-processing (e.g., image segmentation or contour extraction), which can significantly reduce the complexity of the interaction system.

Fingertip Detection and Tracking Using 2D and 3D Information

Single Fingertip Tracking Based on Digiclops Stereo Vision System

Hand Motion Recognition Based on a 3D Fingertip Detection Fusion Method

Real-time Dynamic Gesture Recognition and Hand Servo Tracking Using PTZ Camera

2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

A 3d Fingertips Detecting And Tracking Algorithm Based On The Sliding Window

Robust Fingertip Detection in a Complex Environment.

Real-time Fingertip Detection Based on Depth Data

Real-time Fingertip Tracking and Gesture Recognition Using RGB-D Camera

A Simple 3d Face Tracking Method Based On Depth Information

Fingertip Detection for Human Computer Interaction

Accurate Fingertip Detection from Binocular Mask Images.

Real time fingertip detection using radial symmetry transform

Real Time Fingertip Detection with Kinect Depth Image Sequences.

FINGERTIP DETECTION FOR HUMAN-COMPUTER INTERACTION

Robust Fingertip Tracking with Improved Kalman Filter.

Vision-Based Fingertip Tracking Utilizing Curvature Points Clustering and Hash Model Representation

An effective real-time fingertip positioning system based on gradient information extraction from frame image sequences

Real-time virtual mouse system using RGB-D images and fingertip detection

Two-stream Convolutional Neural Network for Accurate RGB-D Fingertip Detection Using Depth and Edge Information.

Attention Based Dual Branches Fingertip Detection Network and Virtual Key System