Abstract:Hand pose estimation (HPE) plays an important role during the functional assessment of the hand and in potential rehabilitation. It is a challenge to predict the pose of the hand conveniently and accurately during functional tasks, and this limits the application of HPE. In this paper, we propose a novel architecture of a shifted attention regression network (SARN) to perform HPE. Given a depth image, SARN first predicts the spatial relationships between points in the depth image and a group of hand keypoints that determine the pose of the hand. Then, SARN uses these spatial relationships to infer the 3D position of each hand keypoint. To verify the effectiveness of the proposed method, we conducted experiments on three open-source datasets of 3D hand poses: NYU, ICVL, and MSRA. The proposed method achieved state-of-the-art performance with 7.32 mm, 5.91 mm, and 7.17 mm of mean error at the hand keypoints, i.e., mean Euclidean distance between the predicted and ground-truth hand keypoint positions. Additionally, to test the feasibility of SARN in hand movement recognition, a hand movement dataset of 26K depth images from 17 healthy subjects was constructed based on the finger tapping test, an important component of neurological exams administered to Parkinson's patients. Each image was annotated with the tips of the index finger and the thumb. For this dataset, the proposed method achieved a mean error of 2.99 mm at the hand keypoints and comparable performance on three task-specific metrics: the distance, velocity, and acceleration of the relative movement of the two fingertips. Results on the open-source datasets demonstrated the effectiveness of the proposed method, and results on our finger tapping dataset validated its potential for applications in functional task characterization.

A Posture Planning Method in Clustered Synergy Subspace for HIT/DLR Hand II

Hand Posture Recognition and Tracking Based on Bag-of-Words for Human Robot Interaction

Learning Grasp Configuration Through Object-Specific Hand Primitives for Posture Planning of Anthropomorphic Hands

Synergy-based Hand Pose Sensing: Reconstruction Enhancement

Hand posture recognition using finger geometric feature

3D-Freehand-pose Initialization Based on Operator'S Cognitive Behavioral Models

SARN: Shifted Attention Regression Network for 3D Hand Pose Estimation

A Control and Posture Recognition Strategy for Upper-Limb Rehabilitation of Stroke Patients

Machine Learning-Based Hand Pose Generation Using a Haptic Controller

A 3D Hand Attitude Estimation Method for Fixed Hand Posture Based on Dual-View RGB Images

Interactive Hand Pose Estimation: Boosting accuracy in localizing extended finger joints

Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human-Robot Interaction

Calibrating human hand for teleoperating the HIT/DLR hand

Prediction of Initial and Final Postures for Motion Planning in Human Manual Manipulation Tasks Based on Cognitive Decision Making

Task-space Synergies for Reaching using Upper-limb Prostheses

Adaptive Kinematic Modeling for Improved Hand Posture Estimates Using a Haptic Glove

Human-Like Trajectory Planning Based on Postural Synergistic Kernelized Movement Primitives for Robot-Assisted Rehabilitation

Estimation of 3D Human Hand Poses with Structured Pose Prior

Human Posture Segmentation by Constrained Spectral Clustering

Applying 3D Human Hand Pose Estimation to Teleoperation

CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction