Abstract:Hand pose estimation in 3D space from a single RGB image is a highly challenging problem due to self-geometric ambiguities, diverse texture, viewpoints, and self-occlusions. Existing work proves that a network structure with multi-scale resolution subnets, fused in parallel can more effectively shows the spatial accuracy of 2D pose estimation. Nevertheless, the features extracted by traditional convolutional neural networks cannot efficiently express the unique topological structure of hand key points based on discrete and correlated properties. Some applications of hand pose estimation based on traditional convolutional neural networks have demonstrated that the structural similarity between the graph and hand key points can improve the accuracy of the 3D hand pose regression. In this paper, we design and implement an end-to-end network for predicting 3D hand pose from a single RGB image. We first extract multiple feature maps from different resolutions and make parallel feature fusion, and then model a graph-based convolutional neural network module to predict the initial 3D hand key points. Next, we use 2D spatial relationships and 3D geometric knowledge to build a self-supervised module to eliminate domain gaps between 2D and 3D space. Finally, the final 3D hand pose is calculated by averaging the 3D hand poses from the GCN output and the self-supervised module output. We evaluate the proposed method on two challenging benchmark datasets for 3D hand pose estimation. Experimental results show the effectiveness of our proposed method that achieves state-of-the-art performance on the benchmark datasets.

Mask-Pose Cascaded CNN for 2D Hand Pose Estimation from Single Color Image

Cascaded hierarchical CNN for 2D hand pose estimation from a single color image

Multiple-Hand 2D Pose Estimation From a Monocular RGB Image

A hybrid network for estimating 3D interacting hand pose from a single RGB image

Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation

Modelling Human Body Pose for Action Recognition Using Deep Neural Networks

Using a single RGB frame for real time 3D hand pose estimation in the wild

Graph-Based CNNs With Self-Supervised Module for 3D Hand Pose Estimation From Monocular RGB

Mask6D: Masked Pose Priors for 6D Object Pose Estimation.

Hand Pose Estimation via Latent 2.5D Heatmap Regression

PANet: A Pixel-Level Attention Network for 6D Pose Estimation With Embedding Vector Features

Hand3D: Hand Pose Estimation using 3D Neural Network

Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation

Silhouette-Net: 3D Hand Pose Estimation from Silhouettes

Multiple Transfer Net Based Region Ensemble Network for Deep Hand Pose Estimation

Learning to Estimate 3D Human Pose and Shape from a Single Color Image

Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density Network

Region Ensemble Network: Improving Convolutional Network for Hand Pose Estimation

Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation.

Semi-Supervised Joint Learning for Hand Gesture Recognition from a Single Color Image

Recurrent 3D Hand Pose Estimation Using Cascaded Pose-Guided 3D Alignments