Abstract:Gesture and fingertip are becoming more and more important mediums for human-computer interaction (HCI). Therefore, algorithms of gesture recognition and fingertip detection have been extensively investigated. However, problems mainly remain in how to achieve a win-win situation between speed and accuracy, and how to deal with complex interaction environment. To rectify these problems, this paper proposes an attention-based dual branches network that can efficiently fulfill both fingertip detection and gesture recognition tasks. In order to deal with complex interaction environment, we combine both channel-wise attention and spatial-wise attention into the fingertip detection model. The extensive experiments demonstrate that our novel model is both effective and efficient. In the experiment, our proposed model achieves the average fingertip detection error at around 2.8 pixels in 640×480 video frame, and the average recognition accuracy among eight gestures reaches $99%$. Moreover, the average forward time is about 8 ms. Due to the light-weight design, this model can also achieve high-efficiency performance on CPU. In addition, we design a virtual key system based on our proposed model, which can allow users to complete the "clicking" operation naturally in virtual environment. Our proposed system can perform well with a single normal RGB camera without any pre-processing (e.g., image segmentation or contour extraction), which can significantly reduce the complexity of the interaction system.

Single Fingertip Tracking Based on Digiclops Stereo Vision System

Fingertip Detection and Tracking Using 2D and 3D Information

A Single-Camera Gaze Tracking System under Natural Light

Real-time Dynamic Gesture Recognition and Hand Servo Tracking Using PTZ Camera

Human-Computer Interaction System Based on Binocular Stereo Vision

Real-time Fingertip Tracking and Gesture Recognition Using RGB-D Camera

Robust Fingertip Tracking with Improved Kalman Filter.

Real-time virtual mouse system using RGB-D images and fingertip detection

An effective real-time fingertip positioning system based on gradient information extraction from frame image sequences

A New Method of Detecting Fingertip Touch for the Projector-Camera HCI System.

A 3d Fingertips Detecting And Tracking Algorithm Based On The Sliding Window

Accurate Fingertip Detection from Binocular Mask Images.

OneTip: A Soft Tactile Interface for 6-D Fingertip Pose Acquisition in Human-Computer Interaction

Fingertip Detection for Human Computer Interaction

Deep Multi-Task Learning based Fingertip Detection

Attention Based Dual Branches Fingertip Detection Network and Virtual Key System

3D Hand Pose Tracking and Estimation Using Stereo Matching.

13.2: 360° Multi‐Faced Tracking and Interaction Using A Panoramic Camera

Human-Computer Interaction Using Fingertip Based on Kinect

Single Fingertip Detection Using Simple Geometric Properties of the Hand Image: A Case Study for Augmented Reality in an Educational App

Dexterous object manipulation by a multi-fingered robotic hand with visual-tactile fingertip sensors