View Consistent 3D Face Reconstruction Using Siamese Encoder-Decoders

Penglei Ji,Ming Zeng,Xinguo Liu
DOI: https://doi.org/10.1007/978-981-33-6033-4_16
2020-01-01
Abstract:This paper addresses the problem of reconstructing the high-quality 3D face shape and high-fidelity texture from a single image. Traditional methods produce a 3D facial shape consistent with only the viewpoint of the input image. The shape and texture of un-exposed regions are estimated from a classic linear 3D Morphable Model (3DMM), leading to unsatisfactory results. Instead, we propose a novel method which is capable of inferring view-consistent shape and texture in unseen regions. Our method is based on a multi-task encoder-decoder framework which simultaneously learns a coarse 3DMM shape, a texture map and a normal map. To build the underlying nonlinear mapping between the input image and the whole shape/texture, we introduce siamese networks to learn a complete texture map and normal map by utilizing the multi-view images of the same person. Experiments show that our method produces more detailed results compared with the previous methods and can generate consistent and complete facial shape and texture.
What problem does this paper attempt to address?