Abstract:Monocular 3D face reconstruction is a computer vision problem of extraordinary difficulty. Restrictions of large poses and facial details(such as wrinkles, moles, beards etc.) are the common deficiencies of the most existing monocular 3D face reconstruction methods. To resolve the two defects, we propose an end-to-end system to provide 3D reconstructions of faces with details which express robustly under various backgrounds, pose rotations and occlusions. To obtain the facial detail informations, we leverage the image-to-image translation network (we call it p2p-net for short) to make pixel to pixel estimation from the input RGB image to depth map. This precise per-pixel estimation can provide depth value for facial details. And we use a procedure similar to image inpainting to recover the missing details. Simultaneously, for adapting pose rotation and resolving occlusions, we use CNNs to estimate a basic facial model based on 3D Morphable Model(3DMM), which can compensate the unseen facial part in the input image and decrease the deviation of final 3D model by fitting with the dense depth map. We propose an Identity Shape Loss function to enhance the basic facial model and we add a Multi-view Identity Loss that compare the features of the 3D face fusion and the ground truth from multi-view angles. The training data for p2p-net is from 3D scanning system, and we augment the dataset to a larger magnitude for a more generic training. Comparing to other state-of-the-art methods of 3D face reconstruction, we evaluate our method on in-the-wild face images. the qualitative and quantitative comparison show that our method performs both well on robustness and accuracy especially when facing non-frontal pose problems.

Towards Rich-Detail 3D Face Reconstruction and Dense Alignment Via Multi-Scale Detail Augmentation.

High-fidelity 3D Face Reconstruction with Multi-Scale Details

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Joint 3d Face Reconstruction And Dense Face Alignment Via Deep Face Feature Alignment

The face image super-resolution algorithm based on combined representation learning

Face Super-Resolution Via Progressive-Scale Boosting Network

Multi-Scale Feature Fusion and Structure-Preserving Network for Face Super-Resolution

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images

Multi-Scale Spatial Transformation and Attention Based Super-Resolution Reconstruction Algorithm

Total Scale: Face-to-Body Detail Reconstruction from Sparse RGBD Sensors

Dense 3D Face Reconstruction from a Single RGB Image

Fast 3D Face Reconstruction from a Single Image Combining Attention Mechanism and Graph Convolutional Network

CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images.

3D Face Reconstruction Based on ResNet Feature Extraction and CBAM

Weakly-Supervised Multi-Face 3D Reconstruction

A Lightweight Network with Multi-Stage Feature Fusion Module for Single-View 3d Face Reconstruction.

Face Super-Resolution Via Triple-Attention Feature Fusion Network

A Face Structure Attention Network for Face Super-Resolution

Photo-realistic Face Images Synthesis for Learning-based Fine-scale 3D Face Reconstruction.

Replay Attention and Data Augmentation Network for 3D Dense Alignment and Face Reconstruction

Image Reconstruction of Multi Branch Feature Multiplexing Fusion Network with Mixed Multi-layer Attention