Abstract:OBJECTIVE:To establish a deep learning algorithm that can accurately determine three-dimensional facial anatomical landmarks, multi-view stacked hourglass convolutional neural networks (MSH-CNN) and to construct three-dimensional facial midsagittal plane automatically based on MSH-CNN and weighted Procrustes analysis algorithm. METHODS:One hundred subjects with no obvious facial deformity were collected in our oral clinic. Three-dimensional facial data were scanned by three-dimensional facial scanner. Experts annotated twenty-one facial landmarks and midsagittal plane of each data. Eighty three-dimensional facial data were used as training set, to train the MSH-CNN in this study. The overview of MSH-CNN network architecture contained multi-view rendering and training the MSH-CNN network. The three-dimensional facial data were rendered from ninety-six views that were fed to MSH-CNN and the output was one heatmap per landmark. The result of the twenty-one landmarks was accurately placed on the three-dimensional facial data after a three-dimensional view ray voting process. The remaining twenty three-dimensional facial data were used as test set. The trained MSH-CNN automatically determined twenty-one three-dimensional facial anatomical landmarks of each case of data, and calculated the distance between each MSH-CNN landmark and the expert landmark, which was defined as position error. The midsagittal plane of the twenty subjects' could be automatically constructed, using the MSH-CNN and Procrustes analysis algorithm. To evaluate the effect of midsagittal plane by automatic method, the angle between the midsagittal plane constructed by the automatic method and the expert annotated plane was calculated, which was defined as angle error. RESULTS:For twenty subjects with no obvious facial deformity, the average angle error of the midsagittal plane constructed by MSH-CNN and weighted Procrustes analysis algorithm was 0.73°±0.50°, in which the average position error of the twenty-one facial landmarks automatically determined by MSH-CNN was (1.13±0.24) mm, the maximum position error of the orbital area was (1.31±0.54) mm, and the minimum position error of the nasal area was (0.79±0.36) mm. CONCLUSION:This research combines deep learning algorithms and Procrustes analysis algorithms to realize the fully automated construction of the three-dimensional midsagittal plane, which initially achieves the construction effect of clinical experts. The obtained results constituted the basis for the independent intellectual property software development.

3D evaluation model of facial aesthetics based on multi‐input 3D convolution neural networks for orthognathic surgery

Cranio-maxillofacial post-operative face prediction by deep spatial multiband VGG-NET CNN

Quantification of facial symmetry in orthognathic surgery: A novel approach integrating 3D contour maps and hyper-dimensional computing

Deep Learning–Based Facial and Skeletal Transformations for Surgical Planning

Accuracy Evaluation of a Three-Dimensional Face Reconstruction Model Based on the Hifi3D Face Model and Clinical Two-Dimensional Images

Automated 3D Perioral Landmark Detection Using High-Resolution Network: Artificial Intelligence-Based Anthropometric Analysis

Deep Learning-Based Prediction of the 3D Postorthodontic Facial Changes

Artificial Intelligence for Rhinoplasty Design in Asian Patients

Digital Human: Three-dimentional face model reconstruction based on Hifi3dface model with clinical two-dimentional photos

Development of novel artificial intelligence systems to predict facial morphology after orthognathic surgery and orthodontic treatment in Japanese patients

Three-dimensional acquisition technologies for facial soft tissues - Applications and prospects in orthognathic surgery

Deep learning for biomechanical modeling of facial tissue deformation in orthognathic surgical planning

[Deep Learning-Assisted Construction of Three-Demensional Facial Midsagittal Plane].

The Accuracy of a Three-Dimensional Face Model Reconstructing Method Based on Conventional Clinical Two-Dimensional Photos

Application of 3D Imaging-Assisted Precise Aesthetic Evaluation in Midfacial Depression Treatment

Quantifying Nasal Deformities Using a Novel Mathematical Method to Complement Preoperative Assessment in Rhinoplasty Patients

A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning

Automatic Three-Dimensional Cephalometric Annotation System Using Three-Dimensional Convolutional Neural Networks

Research on 3D face measurement for facial virtual plastic surgery

Validating 3D face morphing towards improving pre-operative planning in facial reconstruction surgery

Predicting optimal patient-specific postoperative facial landmarks for patients with craniomaxillofacial deformities