Abstract:Parametric and non-parametric methods are two commonly used strategies in current 3D hand pose reconstruction. Parametric methods predict low-dimensional parameters to fit a predefined hand model to the input image. Benefiting from the prior knowledge of hand models, parametric methods guarantee plausible hand poses, whereas the pose estimation accuracy is limited due to nonlinear regression and spatial information loss. Differently, non-parametric methods directly estimate the coordinates of keypoints or mesh vertices from the input image. The reconstructed 3D hand poses show high precision but may be less robust. In this paper, we integrate the advantages of two methods for accurate and robust hand pose reconstruction. Specifically, we disentangle the hand pose reconstruction into global modeling and local refinement, which is performed in a coarse-to-fine manner. Firstly, we utilize global features from the encoder to generate the initial estimation by a parametric method, which aims to provide the prior knowledge of the human hand for subsequent processes. Then, we gradually fuse multi-scale contextual features for local refinement by explicitly integrating global prior information and local visual features. In particular, we introduce a consecutive pixel-aligned feature retrieval module to extract fine-grained information from visual features, thereby achieving pixel-level alignment. Furthermore, we demonstrate that our method can be extended to weakly-supervised learning where only sparse pose annotations are needed, potentially alleviating the burden of meticulous mesh annotation. The effectiveness and robustness of our method are substantiated through both fully- and weakly-supervised experiments, demonstrating superior performance compared to state-of-the-art methods. We plan to release our code at https://github.com/Kun-Gao/P_GLFnet.

BlendPose: A New Linear Parametric Hand Model for High-quality 3D Hand Reconstruction

Personalized Hand Modeling from Multiple Postures with Multi‐View Color Images

HandFormer: Hand Pose Reconstructing from a Single RGB Image

Progressively Global-Local Fusion with Explicit Guidance for Accurate and Robust 3d Hand Pose Reconstruction

RealisticHands: A Hybrid Model for 3D Hand Reconstruction

Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering

3D Hand Reconstruction via Aggregating Intra and Inter Graphs Guided by Prior Knowledge for Hand-Object Interaction Scenario

3D Hand Pose Estimation and Reconstruction Based on Multi-Feature Fusion

A Simple Baseline for Efficient Hand Mesh Reconstruction

Efficient 3D Scan Registration for Hand Modeling and Reconstruction

Overcoming the Trade-off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction

BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks

UniHands: Unifying Various Wild-Collected Keypoints for Personalized Hand Reconstruction

HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture

Hand-3d-Studio: A New Multi-View System for 3d Hand Reconstruction.

Interacting Two-Hand 3D Pose and Shape Reconstruction from Single Color Image

3D Hand Pose and Shape Estimation from Monocular RGB Via Efficient 2D Cues

HandO: a Hybrid 3D Hand–object Reconstruction Model for Unknown Objects

SMR: Spatial-Guided Model-Based Regression for 3D Hand Pose and Mesh Reconstruction

3D Hand Reconstruction from a Single Image Based on Biomechanical Constraints

HandOS: 3D Hand Reconstruction in One Stage