Fully automated landmarking and facial segmentation on 3D photographs

Bo Berends,Freek Bielevelt,Ruud Schreurs,Shankeeth Vinayahalingam,Thomas Maal,Guido de Jong

DOI: https://doi.org/10.1038/s41598-024-56956-9

IF: 4.6

2024-03-19

Scientific Reports

Abstract:Three-dimensional facial stereophotogrammetry provides a detailed representation of craniofacial soft tissue without the use of ionizing radiation. While manual annotation of landmarks serves as the current gold standard for cephalometric analysis, it is a time-consuming process and is prone to human error. The aim in this study was to develop and evaluate an automated cephalometric annotation method using a deep learning-based approach. Ten landmarks were manually annotated on 2897 3D facial photographs. The automated landmarking workflow involved two successive DiffusionNet models. The dataset was randomly divided into a training and test dataset. The precision of the workflow was evaluated by calculating the Euclidean distances between the automated and manual landmarks and compared to the intra-observer and inter-observer variability of manual annotation and a semi-automated landmarking method. The workflow was successful in 98.6% of all test cases. The deep learning-based landmarking method achieved precise and consistent landmark annotation. The mean precision of 1.69 ± 1.15 mm was comparable to the inter-observer variability (1.31 ± 0.91 mm) of manual annotation. Automated landmark annotation on 3D photographs was achieved with the DiffusionNet-based approach. The proposed method allows quantitative analysis of large datasets and may be used in diagnosis, follow-up, and virtual surgical planning.

multidisciplinary sciences

What problem does this paper attempt to address?

This paper attempts to address the problem of automatically annotating soft tissue anatomical landmarks in three-dimensional (3D) facial photographs. Currently, manually annotating these landmarks is the gold standard for cephalometric analysis, but this method is time-consuming and prone to errors. Therefore, researchers have developed and evaluated a deep learning-based approach to achieve automatic cephalometric annotation in 3D facial photographs. Specifically, the goals of this study include: 1. **Developing an automated method**: Using deep learning techniques (specifically the DiffusionNet model) to automatically annotate anatomical landmarks in 3D facial photographs. 2. **Evaluating accuracy**: Assessing the accuracy and consistency of the automated method by comparing it with manual annotation and semi-automated methods. 3. **Improving efficiency**: Reducing the time and labor costs of manual annotation through automation, thereby enabling quantitative analysis of large datasets in clinical diagnosis, follow-up, and virtual surgical planning. The researchers selected 10 common facial anatomical landmarks for annotation and randomly divided the dataset into training and testing sets to validate the method's effectiveness. The results indicate that the automated method can achieve accuracy comparable to manual annotation in most cases, with an average accuracy of 1.69 ± 1.15 millimeters, close to the inter-observer variability of manual annotation (1.31 ± 0.91 millimeters).

Fully automated landmarking and facial segmentation on 3D photographs

Fully automated landmarking and facial segmentation on 3D photographs

A Cross-Dimension Annotations Method for 3D Structural Facial Landmark Extraction

Learning-based local-to-global landmark annotation for automatic 3D cephalometry

Automatic landmark annotation and dense correspondence registration for 3D human facial images

Automated Cephalometric Landmark Detection: A Novel Software Model Compared with Manual Annotation Method

An Automated Method of 3D Facial Soft Tissue Landmark Prediction Based on Object Detection and Deep Learning

Deep learning for 3D cephalometric landmarking with heterogeneous multi-center CBCT dataset

Automated 3D Perioral Landmark Detection Using High-Resolution Network: Artificial Intelligence-Based Anthropometric Analysis

CephalFormer: Incorporating Global Structure Constraint into Visual Features for General Cephalometric Landmark Detection

Deep Learning Techniques for Automatic Lateral X-ray Cephalometric Landmark Detection: Is the Problem Solved?

Automated detection of cephalometric landmarks using deep neural patchworks

Automatic craniomaxillofacial landmarks detection in CT images of individuals with dentomaxillofacial deformities by a two-stage deep learning model

Automatic Three-Dimensional Cephalometric Annotation System Using Three-Dimensional Convolutional Neural Networks

CephaNet: an Improved Faster R-CNN for Cephalometric Landmark Detection.

Accuracy and reliability of automated three-dimensional facial landmarking in medical and biological studies. A systematic review

A Registration and Deep Learning Approach to Automated Landmark Detection for Geometric Morphometrics

Automatic Cephalometric Landmark Identification System Based on the Multi-Stage Convolutional Neural Networks with CBCT Combination Images

Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognition.

Automated 3D Landmarking of the Skull: A Novel Approach for Craniofacial Analysis

Deep learning for cephalometric landmark detection: systematic review and meta-analysis