SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction

Yukang Cao,Kai Han,Kwan-Yee K. Wong

2023-04-02

Abstract:We address the problem of clothed human reconstruction from a single image or uncalibrated multi-view images. Existing methods struggle with reconstructing detailed geometry of a clothed human and often require a calibrated setting for multi-view reconstruction. We propose a flexible framework which, by leveraging the parametric SMPL-X model, can take an arbitrary number of input images to reconstruct a clothed human model under an uncalibrated setting. At the core of our framework is our novel self-evolved signed distance field (SeSDF) module which allows the framework to learn to deform the signed distance field (SDF) derived from the fitted SMPL-X model, such that detailed geometry reflecting the actual clothed human can be encoded for better reconstruction. Besides, we propose a simple method for self-calibration of multi-view images via the fitted SMPL-X parameters. This lifts the requirement of tedious manual calibration and largely increases the flexibility of our method. Further, we introduce an effective occlusion-aware feature fusion strategy to account for the most useful features to reconstruct the human model. We thoroughly evaluate our framework on public benchmarks, demonstrating significant superiority over the state-of-the-arts both qualitatively and quantitatively.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to reconstruct the 3D model of a clothed human body from a single image or uncalibrated multi - view images. Existing methods have difficulties in reconstructing the detailed geometric structure of a clothed human body and usually require a calibration setting in multi - view reconstruction. In addition, existing methods perform poorly when dealing with non - frontal self - occluded human postures, and multi - view reconstruction methods rely on camera calibration, which is very cumbersome in practical applications. Therefore, this paper proposes a flexible framework. By using the parameterized SMPL - X model as a shape prior, it can reconstruct the clothed human body model from any number of input images in an uncalibrated setting. The core innovation lies in the introduction of the Self - Evolving Signed Distance Field (SeSDF) module. This module can learn to deform the Signed Distance Field (SDF) derived from the SMPL - X model to encode more detailed geometric information, thus achieving better reconstruction results. In addition, a self - calibration method for multi - view images and an effective occlusion - aware feature fusion strategy are also proposed to improve the robustness of multi - view reconstruction.