Abstract:Touchscreens have become the primary input devices for smartphones, tablet computers, and other intelligent devices over the past decades. While for the most pervasive commercial devices, only 2D touch positions on the screen are utilized as interaction inputs. To extend the richness of the input vocabulary, some researchers have proposed several innovative interaction techniques, e.g. finger pose. However, due to the low resolution and lacking in information of capacitive images, only two angles, pitch and yaw, are considered in most finger pose estimation algorithms, and the accuracy is not sufficiently high for large scale applications in smartphones. With the rapid development of under-screen fingerprint sensing technology, a new input modality, fingerprint image, for 3D finger pose estimation is available from these fingerprint sensors. In this paper, we propose a finger specific algorithm for estimating 3D finger pose including roll, pitch, and yaw from fingerprint images. 3D finger surface is first reconstructed based on sequential fingerprint images captured in enrollment, and given this 3D surface model, 3D finger pose of a test fingerprint is estimated by matching keypoints between the 2D image and 3D point cloud and minimizing the projection error. The proposed approach is a non-learning algorithm with good generalization ability and robustness in real applications. To evaluate the performance of our method, a dataset of fingerprint images with their corresponding ground truth 3D angles is collected. Experimental results on this dataset demonstrate the effectiveness of introducing reconstructed 3D finger surface shape in 3D finger pose estimation. The average absolute errors of three angles are 10.74 for roll, 8.25 for pitch, and 7.38 for yaw, respectively. Extensive experiments are also conducted to explore the impact of touching area size and gallery size on performance.

Accurate Fingertip Detection from Binocular Mask Images.

Accurate Iris Center Localization Method Using Facial Landmark, Snakuscule, Circle Fitting and Binary Connected Component

Two-Stream Binocular Network: Accurate Near Field Finger Detection Based On Binocular Images

Two-stream Convolutional Neural Network for Accurate RGB-D Fingertip Detection Using Depth and Edge Information.

Fingertip Detection and Tracking Using 2D and 3D Information

Fingertip in the Eye: A cascaded CNN pipeline for the real-time fingertip detection in egocentric videos

Deep Multi-Task Learning based Fingertip Detection

Attention Based Dual Branches Fingertip Detection Network and Virtual Key System

A New Method of Detecting Fingertip Touch for the Projector-Camera HCI System.

Real-time Fingertip Tracking and Gesture Recognition Using RGB-D Camera

Kinect Sensor-Based Long-Distance Hand Gesture Recognition and Fingertip Detection with Depth Information

Faster R-CNN with Inception V2 for Fingertip Detection in Homogenous Background Image

Robust Fingertip Tracking with Improved Kalman Filter.

Tracking of Fingertips and Centres of Palm using KINECT

Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network for Capturing Hand Articulations

Estimating 3D Finger Pose Via 2D-3D Fingerprint Matching

Joint Identity Verification and Pose Alignment for Partial Fingerprints

Efficient and Accurate 3D Finger Knuckle Matching using Surface Key Points

Hand Segmentation and Fingertip Tracking from Depth Camera Images Using Deep Convolutional Neural Network and Multi-task SegNet

A Novel Multicamera System for High-Speed Touchless Palm Recognition

FKIMNet: A Finger Dorsal Image Matching Network Comparing Component (Major, Minor and Nail) Matching with Holistic (Finger Dorsal) Matching