From Skulls to Faces: A Deep Generative Framework for Realistic 3D Craniofacial Reconstruction.

Yehong Pan,Jian Wang,Guihong Liu,Qiushuo Wu,Yazi Zheng,Xin Lan,Weibo Liang,Jiancheng Lv,Yuan Li

DOI: https://doi.org/10.1007/978-3-031-53305-1_24

2024-01-01

Abstract:The shape of the human face is largely determined by the underlying skull morphology. Craniofacial reconstruction (CfR), or the process of reconstructing the face from the skull, is a challenging task with applications in forensic science, criminal investigation, and archaeology. Traditional craniofacial reconstruction methods suffer from subjective interpretation and simple low-dimensional learning approaches, resulting in low reconstruction accuracy and realism. In this paper, we present a deep learning-based framework for CfR based on conditional generative adversarial networks. Unlike conventional methods that adopt 3D representations directly, we employ 2D depth maps to represent faces and skulls as the model's input and output. It can provide enough face geometric information and may mitigate the potential risk of dimensionality issues. Our framework is capable of modeling both local and global details of facial appearance through a novel discriminator structure that leverages multi-receptive field features in one output, thus generating realistic and individualized faces from skulls. Furthermore, to explore the impact of conditional information such as age and gender on facial appearance, we develop a conditional CfR paradigm that incorporates an improved residual block structure with conditional information modulation and a conditional information reconstruction loss function. Extensive experiments and comparisons are conducted to demonstrated the effectiveness and superiority of our method for CfR.