Abstract:Gesture and fingertip are becoming more and more important mediums for human-computer interaction (HCI). Therefore, algorithms of gesture recognition and fingertip detection have been extensively investigated. However, problems mainly remain in how to achieve a win-win situation between speed and accuracy, and how to deal with complex interaction environment. To rectify these problems, this paper proposes an attention-based dual branches network that can efficiently fulfill both fingertip detection and gesture recognition tasks. In order to deal with complex interaction environment, we combine both channel-wise attention and spatial-wise attention into the fingertip detection model. The extensive experiments demonstrate that our novel model is both effective and efficient. In the experiment, our proposed model achieves the average fingertip detection error at around 2.8 pixels in 640×480 video frame, and the average recognition accuracy among eight gestures reaches $99%$. Moreover, the average forward time is about 8 ms. Due to the light-weight design, this model can also achieve high-efficiency performance on CPU. In addition, we design a virtual key system based on our proposed model, which can allow users to complete the "clicking" operation naturally in virtual environment. Our proposed system can perform well with a single normal RGB camera without any pre-processing (e.g., image segmentation or contour extraction), which can significantly reduce the complexity of the interaction system.

A Fingertips Detection Method Based On The Combination Of Centroid And Harris Corner Algorithm

Accurate Iris Center Localization Method Using Facial Landmark, Snakuscule, Circle Fitting and Binary Connected Component

A real-time hand gesture recognition method

Multimodal Biometric Identification System Based on Finger Geometry, Knuckle Print and Palm Print

Efficient Hand Segmentation and Fingertip Detection Using Color Features of Skin and Fingernail.

A New Touchless Palmprint Location Method Based on Contour Centroid

Fingertip Detection and Tracking Using 2D and 3D Information

Attention Based Dual Branches Fingertip Detection Network and Virtual Key System

Deep Multi-Task Learning based Fingertip Detection

Two-stream Convolutional Neural Network for Accurate RGB-D Fingertip Detection Using Depth and Edge Information.

Accurate Fingertip Detection from Binocular Mask Images.

FINGERTIP DETECTION FOR HUMAN-COMPUTER INTERACTION

Two-Stream Binocular Network: Accurate Near Field Finger Detection Based On Binocular Images

A New Method of Detecting Fingertip Touch for the Projector-Camera HCI System.

Human detection based on CENTRIST and scale of edge selection

Faster R-CNN with Inception V2 for Fingertip Detection in Homogenous Background Image

Robust Hybrid Finger Pattern Identification Using Intersection Enhanced Gabor Based Direction Coding

A Finger-Touching Detection Method Using Stripes and Shadow

Fingertip Detection Method in Hand Gesture Recognition

Hand Gesture Recognition Using Appearance Features Based on 3D Point Cloud

Kinect Sensor-Based Long-Distance Hand Gesture Recognition and Fingertip Detection with Depth Information