Abstract:Hand pose estimation (HPE) plays an important role during the functional assessment of the hand and in potential rehabilitation. It is a challenge to predict the pose of the hand conveniently and accurately during functional tasks, and this limits the application of HPE. In this paper, we propose a novel architecture of a shifted attention regression network (SARN) to perform HPE. Given a depth image, SARN first predicts the spatial relationships between points in the depth image and a group of hand keypoints that determine the pose of the hand. Then, SARN uses these spatial relationships to infer the 3D position of each hand keypoint. To verify the effectiveness of the proposed method, we conducted experiments on three open-source datasets of 3D hand poses: NYU, ICVL, and MSRA. The proposed method achieved state-of-the-art performance with 7.32 mm, 5.91 mm, and 7.17 mm of mean error at the hand keypoints, i.e., mean Euclidean distance between the predicted and ground-truth hand keypoint positions. Additionally, to test the feasibility of SARN in hand movement recognition, a hand movement dataset of 26K depth images from 17 healthy subjects was constructed based on the finger tapping test, an important component of neurological exams administered to Parkinson's patients. Each image was annotated with the tips of the index finger and the thumb. For this dataset, the proposed method achieved a mean error of 2.99 mm at the hand keypoints and comparable performance on three task-specific metrics: the distance, velocity, and acceleration of the relative movement of the two fingertips. Results on the open-source datasets demonstrated the effectiveness of the proposed method, and results on our finger tapping dataset validated its potential for applications in functional task characterization.

Residual Attention Regression For 3d Hand Pose Estimation

Dual Regression for Efficient Hand Pose Estimation

Attention Residual Network with 3D convolutional neural network for 3D Human Pose Estimation.

Attention-Based Pose Sequence Machine for 3D Hand Pose Estimation

Spatial-aware Stacked Regression Network for Real-Time 3D Hand Pose Estimation.

Learning Hand Latent Features for Unsupervised 3D Hand Pose Estimation

Pixel-wise Regression: 3D Hand Pose Estimation via Spatial-form Representation and Differentiable Decoder

Accurate 3D Hand Pose Estimation Network Utilizing Joints Information.

Recurrent 3D Hand Pose Estimation Using Cascaded Pose-Guided 3D Alignments

AWR: Adaptive Weighting Regression for 3D Hand Pose Estimation

Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks

3D Human Pose Estimation in Motion Based on Multi-Stage Regression

NETWORKS EFFECTIVELY UTILIZING 2D SPATIAL INFORMATION FOR ACCURATE 3D HAND POSE ESTIMATION

Calibrated deep attention model for 3D pose estimation in the wild

DOR3D-Net: Dense Ordinal Regression Network for 3D Hand Pose Estimation

Hand Pose Estimation with Attention-and-Sequence Network.

Hand3D: Hand Pose Estimation using 3D Neural Network

HMTNet:3D Hand Pose Estimation from Single Depth Image Based on Hand Morphological Topology

Differentiable Spatial Regression: A Novel Method for 3D Hand Pose Estimation.

3D Hand Pose Estimation Using Synthetic Data and Weakly Labeled RGB Images.

SARN: Shifted Attention Regression Network for 3D Hand Pose Estimation