Abstract:The authors discover the unbalanced nature of existing clothed human reconstruction approaches and propose the balanced implicit function (BIF) for robust 3D clothed human reconstruction in diverse poses and clothing. There are two keys: (1) Using a novel dependency coefficient to synthesise balanced training samples; (2) Designing a novel positional embedding‐based attenuation strategy to reduce the interference from unreliable body priors. The proposed method significantly surpasses state‐of‐the‐art approaches and generalises well on unseen poses and loose clothing. The authors study the problem of reconstructing detailed 3D human surfaces in various poses and clothing from images. The parametric human body allows accurate 3D clothed human reconstruction. However, the offset of large and loose clothing from the inferred parametric body mesh confines the generalisation of the existing parametric body‐based methods. A distinctive method that simultaneously generalises well to unseen poses and unseen clothing is proposed. The authors first discover the unbalanced nature of existing implicit function‐based methods. To address this issue, the authors propose to synthesise the balanced training samples with a new dependency coefficient in training. The dependency coefficient can tell the network whether the prior from the parametric body model is reliable. The authors then design a novel positional embedding‐based attenuation strategy to incorporate the dependency coefficient into the implicit function (IF) network. Comprehensive experiments are conducted on the CAPE dataset to study the effectiveness of the authors' approach. The proposed method significantly surpasses state‐of‐the‐art approaches and generalises well on unseen poses and clothing. As an illustrative example, the proposed method improves the Chamfer Distance Error and Normal Error by 38.2% and 57.6%.

Implicit 3D Human Reconstruction Guided by Parametric Models and Normal Maps

RICH: Robust Implicit Clothed Humans Reconstruction from Multi-scale Spatial Cues.

ECON: Explicit Clothed humans Optimized via Normal integration

HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction

Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction

Balanced parametric body prior for implicit clothed human reconstruction from a monocular RGB

SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

Single-view 3D Body and Cloth Reconstruction under Complex Poses

HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models

High-Resolution Volumetric Reconstruction for Clothed Humans

3D Human Reconstruction from A Single Depth Image

Pixel2ISDF: Implicit Signed Distance Fields based Human Body Model from Multi-view and Multi-pose Images

Innovative AI techniques for photorealistic 3D clothed human reconstruction from monocular images or videos: a survey

Topology-Preserved Human Reconstruction with Details

PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction

Disambiguating Monocular Reconstruction of 3D Clothed Human with Spatial-Temporal Transformer

Multi‐Level Implicit Function for Detailed Human Reconstruction by Relaxing SMPL Constraints

PSHuman: Photorealistic Single-view Human Reconstruction using Cross-Scale Diffusion

SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction

ReN Human: Learning Relightable Neural Implicit Surfaces for Animatable Human Rendering