Abstract:Touchscreens have become the primary input devices for smartphones, tablet computers, and other intelligent devices over the past decades. While for the most pervasive commercial devices, only 2D touch positions on the screen are utilized as interaction inputs. To extend the richness of the input vocabulary, some researchers have proposed several innovative interaction techniques, e.g. finger pose. However, due to the low resolution and lacking in information of capacitive images, only two angles, pitch and yaw, are considered in most finger pose estimation algorithms, and the accuracy is not sufficiently high for large scale applications in smartphones. With the rapid development of under-screen fingerprint sensing technology, a new input modality, fingerprint image, for 3D finger pose estimation is available from these fingerprint sensors. In this paper, we propose a finger specific algorithm for estimating 3D finger pose including roll, pitch, and yaw from fingerprint images. 3D finger surface is first reconstructed based on sequential fingerprint images captured in enrollment, and given this 3D surface model, 3D finger pose of a test fingerprint is estimated by matching keypoints between the 2D image and 3D point cloud and minimizing the projection error. The proposed approach is a non-learning algorithm with good generalization ability and robustness in real applications. To evaluate the performance of our method, a dataset of fingerprint images with their corresponding ground truth 3D angles is collected. Experimental results on this dataset demonstrate the effectiveness of introducing reconstructed 3D finger surface shape in 3D finger pose estimation. The average absolute errors of three angles are 10.74 for roll, 8.25 for pitch, and 7.38 for yaw, respectively. Extensive experiments are also conducted to explore the impact of touching area size and gallery size on performance.

Joint Estimation of Pose and Singular Points of Fingerprints

Joint Identity Verification and Pose Alignment for Partial Fingerprints

Fingerprint pose estimation based on faster R-CNN

Fingerprint Indexing with Pose Constraint.

Estimating Fingerprint Pose Via Dense Voting.

Accurate 3D Hand Pose Estimation Network Utilizing Joints Information.

Joint Multi-Person Pose Estimation and Semantic Part Segmentation

Estimating 3D Finger Pose Via 2D-3D Fingerprint Matching

Orientation Field Estimation for Latent Fingerprints by Exhaustive Search of Large Database

Hand Pose Estimation with Attention-and-Sequence Network.

Interactive Hand Pose Estimation: Boosting accuracy in localizing extended finger joints

Bi-Stream Pose-Guided Region Ensemble Network for Fingertip Localization From Stereo Images

Pose-Specific 3D Fingerprint Unfolding

Simultaneous 3D Hand Detection and Pose Estimation Using Single Depth Images

A novel joint points and silhouette-based method to estimate 3D human pose and shape

Reconstructing 3D human pose and shape from a single image and sparse IMUs

Improving Latent Fingerprint Orientation Field Estimation Using Inpainting Techniques

Human Pose Estimation Based on Human Limbs.

Joint Hand-Object Pose Estimation with Differentiably-Learned Physical Contact Point Analysis

Model Based Algorithm For Singular Point Detection From Fingerprint Images

Joint Hand Detection and Rotation Estimation by Using CNN