Abstract:Recent techniques on implicit geometry representation learning and neural rendering have shown promising results for 3D clothed human reconstruction from sparse video inputs. However, it is still challenging to reconstruct detailed surface geometry and even more difficult to synthesize photorealistic novel views with animated human poses. In this work, we introduce PGAHum, a prior-guided geometry and appearance learning framework for high-fidelity animatable human reconstruction. We thoroughly exploit 3D human priors in three key modules of PGAHum to achieve high-quality geometry reconstruction with intricate details and photorealistic view synthesis on unseen poses. First, a prior-based implicit geometry representation of 3D human, which contains a delta SDF predicted by a tri-plane network and a base SDF derived from the prior SMPL model, is proposed to model the surface details and the body shape in a disentangled manner. Second, we introduce a novel prior-guided sampling strategy that fully leverages the prior information of the human pose and body to sample the query points within or near the body surface. By avoiding unnecessary learning in the empty 3D space, the neural rendering can recover more appearance details. Last, we propose a novel iterative backward deformation strategy to progressively find the correspondence for the query point in observation space. A skinning weights prediction model is learned based on the prior provided by the SMPL model to achieve the iterative backward LBS deformation. Extensive quantitative and qualitative comparisons on various datasets are conducted and the results demonstrate the superiority of our framework. Ablation studies also verify the effectiveness of each scheme for geometry and appearance learning.

Efficient Human Rendering with Geometric and Semantic Priors.

Semantic-Human: Neural Rendering of Humans from Monocular Video with Human Parsing

EG-HumanNeRF: Efficient Generalizable Human NeRF Utilizing Human Prior for Sparse View

ReN Human: Learning Relightable Neural Implicit Surfaces for Animatable Human Rendering

Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

Geometry-guided generalizable NeRF for human rendering

HumanGen: Generating Human Radiance Fields with Explicit Priors

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

SemanticHuman-HD: High-Resolution Semantic Disentangled 3D Human Generation

Point-Based Neural Scene Rendering for Street Views

HDhuman: High-quality Human Novel-view Rendering from Sparse Views

Semantic-based user interests computation

PGAHum: Prior-Guided Geometry and Appearance Learning for High-Fidelity Animatable Human Reconstruction

Learning Compositional Radiance Fields of Dynamic Human Heads

Efficient Meshy Neural Fields for Animatable Human Avatars

R2Human: Real-Time 3D Human Appearance Rendering from a Single Image

Generalizable Neural Radiance Field with Hierarchical Geometry Constraint.

HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs

Prior-Driven NeRF: Prior Guided Rendering

OccFusion: Rendering Occluded Humans with Generative Diffusion Priors

Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatar