Abstract:Recent techniques on implicit geometry representation learning and neural rendering have shown promising results for 3D clothed human reconstruction from sparse video inputs. However, it is still challenging to reconstruct detailed surface geometry and even more difficult to synthesize photorealistic novel views with animated human poses. In this work, we introduce PGAHum, a prior-guided geometry and appearance learning framework for high-fidelity animatable human reconstruction. We thoroughly exploit 3D human priors in three key modules of PGAHum to achieve high-quality geometry reconstruction with intricate details and photorealistic view synthesis on unseen poses. First, a prior-based implicit geometry representation of 3D human, which contains a delta SDF predicted by a tri-plane network and a base SDF derived from the prior SMPL model, is proposed to model the surface details and the body shape in a disentangled manner. Second, we introduce a novel prior-guided sampling strategy that fully leverages the prior information of the human pose and body to sample the query points within or near the body surface. By avoiding unnecessary learning in the empty 3D space, the neural rendering can recover more appearance details. Last, we propose a novel iterative backward deformation strategy to progressively find the correspondence for the query point in observation space. A skinning weights prediction model is learned based on the prior provided by the SMPL model to achieve the iterative backward LBS deformation. Extensive quantitative and qualitative comparisons on various datasets are conducted and the results demonstrate the superiority of our framework. Ablation studies also verify the effectiveness of each scheme for geometry and appearance learning.

GRPose: Learning Graph Relations for Human Image Generation with Pose Priors

GFPose: Learning 3D Human Pose Prior with Gradient Fields

Diffusion-Based Pose Refinement and Multi-Hypothesis Generation for 3D Human Pose Estimation

Pose Generator ( G ) : Head : R arm : L arm : Chest : R leg : L leg Plausible Pose

Diffusion-based Pose Refinement and Muti-hypothesis Generation for 3D Human Pose Estimaiton

An adversarial human pose estimation network injected with graph structure

Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation

Structure-aware human pose estimation with graph convolutional networks

RePoseDM: Recurrent Pose Alignment and Gradient Guidance for Pose Guided Image Synthesis

PGAHum: Prior-Guided Geometry and Appearance Learning for High-Fidelity Animatable Human Reconstruction

PoseGraphNet++: Enriching 3D Human Pose with Orientation Estimation

Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priors

GLPose: Global-Local Representation Learning for Human Pose Estimation

Part-Preserving Pose Manipulation For Person Image Synthesis

Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation

Hierarchical Graph Networks for 3D Human Pose Estimation

G2O-Pose: Real-Time Monocular 3D Human Pose Estimation Based on General Graph Optimization

3D Human Pose Estimation with Generative Adversarial Networks

DPoser: Diffusion Model as Robust 3D Human Pose Prior

Personalized Graph Generation for Monocular 3D Human Pose and Shape Estimation

PoseGU: 3D Human Pose Estimation with Novel Human Pose Generator and Unbiased Learning