High-Fidelity Human Avatars from a Single RGB Camera

Hao Zhao,Jinsong Zhang,Yu-Kun Lai,Zerong Zheng,Yingdi Xie,Yebin Liu,Kun Li
DOI: https://doi.org/10.1109/cvpr52688.2022.01544
2022-01-01
Abstract:In this paper, we propose a coarse-to-fine framework to reconstruct a personalized high-fidelity human avatar from a monocular video. To deal with the misalignment problem caused by the changed poses and shapes in different frames, we design a dynamic surface network to recover pose-dependent surface deformations, which help to decouple the shape and texture of the person. To cope with the complexity of textures and generate photo-realistic results, we propose a reference-based neural rendering network and exploit a bottom-up sharpening-guided fine-tuning strategy to obtain detailed textures. Our frame-work also enables photo-realistic novel view/pose syn-thesis and shape editing applications. Experimental re-sults on both the public dataset and our collected dataset demonstrate that our method outperforms the state-of-the-art methods. The code and dataset will be available at http://cic.tju.edu.cn/faculty/likun/projects/HF-Avatar.
What problem does this paper attempt to address?