3D Facial Imperfection Regeneration: Deep learning approach and 3D printing prototypes

Phuong D. Nguyen,Thinh D. Le,Duong Q. Nguyen,Thanh Q. Nguyen,Li-Wei Chou,H. Nguyen-Xuan

2023-03-25

Abstract:This study explores the potential of a fully convolutional mesh autoencoder model for regenerating 3D nature faces with the presence of imperfect areas. We utilize deep learning approaches in graph processing and analysis to investigate the capabilities model in recreating a filling part for facial scars. Our approach in dataset creation is able to generate a facial scar rationally in a virtual space that corresponds to the unique circumstances. Especially, we propose a new method which is named 3D Facial Imperfection Regeneration(3D-FaIR) for reproducing a complete face reconstruction based on the remaining features of the patient face. To further enhance the applicable capacity of the present research, we develop an improved outlier technique to separate the wounds of patients and provide appropriate wound cover models. Also, a Cir3D-FaIR dataset of imperfect faces and open codes was released at <a class="link-external link-https" href="https://github.com/SIMOGroup/3DFaIR" rel="external noopener nofollow">this https URL</a>. Our findings demonstrate the potential of the proposed approach to help patients recover more quickly and safely through convenient techniques. We hope that this research can contribute to the development of new products and innovative solutions for facial scar regeneration.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper is primarily dedicated to addressing the issue of facial scar regeneration. Specifically, it attempts to reconstruct 3D facial models with imperfect areas (such as scars) using deep learning methods. The main objectives of the research include:

1. **Developing a fully automated method**: To overcome the inaccuracies and inconsistencies in manual filling and sculpting processes, the paper proposes a method based on a Fully Convolutional Mesh Autoencoder to automatically reconstruct facial defect areas.

2. **Creating a virtual dataset**: Due to ethical considerations limiting the use of real patient facial scar images, the research team constructed a dataset named Cir3D-FaIR. This dataset was created using virtual space generation technology to produce a series of facial models with different features and scars, protecting patient privacy while being used to train machine learning models.

3. **Proposing 3D Facial Imperfection Regeneration (3D-FaIR)**: This is a new method aimed at reconstructing a complete facial model based on the remaining features of the patient's face. This method not only improves the accuracy and safety of the reconstruction but also potentially promotes faster patient recovery.

4. **Improved outlier technique**: To optimize model performance, the paper also proposes an improved outlier handling technique to better separate the patient's wounds and provide appropriate coverage models for the wounds.

5. **Practical application-oriented**: The ultimate goal is to transform these reconstructed models into printable 3D prototypes to assist in clinical practice, thereby improving patient treatment outcomes and quality of life.

In summary, this paper focuses on proposing a novel, automated method to address the issue of facial scar reconstruction. By applying deep learning and 3D printing technologies, it aims to provide patients with more efficient and accurate treatment solutions.