Abstract:The computer-assisted radiologic informative report has received increasing research attention to facilitate diagnosis and treatment planning for dental care providers. However, manual interpretation of dental images is limited, expensive, and time-consuming. Another barrier in dental imaging is the limited number of available images for training, which is a challenge in the era of deep learning. This study proposes a novel self-distillation (SD) enhanced self-supervised learning on top of the masked image modeling (SimMIM) Transformer, called SD-SimMIM, to improve the outcome with a limited number of dental radiographs. In addition to the prediction loss on masked patches, SD-SimMIM computes the self-distillation loss on the visible patches. We apply SD-SimMIM on dental panoramic X-rays for teeth numbering, detection of dental restorations and orthodontic appliances, and instance segmentation tasks. Our results show that SD-SimMIM outperforms other self-supervised learning methods. Furthermore, we augment and improve the annotation of an existing dataset of panoramic X-rays.

What problem does this paper attempt to address?

This paper aims to solve several key problems in dental panoramic radiograph analysis, as follows: 1. **Limitations of Manual Interpretation of Dental Images**: Manual interpretation of dental images has the problems of low efficiency, high cost and long time - consuming. This limits the application of dental images in diagnosis and treatment planning. 2. **Limited Training Data**: The dental image data set is relatively small, which is a challenge in the era of deep learning, because deep - learning models usually need a large amount of data for training to achieve the best performance. To solve these problems, the paper proposes a new method named SD - SimMIM, that is, self - distillation - enhanced self - supervised learning method, based on Masked Image Modeling (MIM). Through the self - distillation technique, this method uses the knowledge of visible patches in the decoder to guide the encoder's learning, thereby improving the performance of the model with limited data. ### Main Contributions 1. **Proposing SD - SimMIM**: This is a self - distillation - enhanced SimMIM method, aiming to improve the feature representation ability, reduce the need for large - scale data, and further assist downstream tasks. 2. **Expanding the Data Set**: The authors add the annotations of orthodontic appliances, including brackets, bands and retainers, on the basis of the existing data set, forming a high - quality enhanced data set called Dentalysis annotations. ### Method Overview - **SimMIM Framework**: - **Patchifying and Masking**: Divide the input image into multiple patches and randomly select a part for masking. - **Encoder**: The encoder receives the unmasked patches and extracts latent features. - **Decoder**: The decoder receives the encoded features and learns low - level representations to reconstruct the image. - **Prediction Target**: Define the prediction target and calculate the loss between the predicted value and the actual value in the masked area. - **Self - distillation**: - Through the self - distillation technique, the knowledge of visible patches in the decoder is transferred to the encoder, further enhancing the feature representation ability of the model. - Calculate the self - distillation loss and combine it with the prediction loss in the masked area to form the total loss function. ### Experimental Results - **Quantitative Results**: On the tasks of tooth numbering, dental prosthesis detection and instance segmentation, SD - SimMIM shows better performance than other self - supervised learning methods. - **Qualitative Results**: Through the visualization of image reconstruction and detection results, the improvement effect of SD - SimMIM on image reconstruction and detection tasks is demonstrated. ### Conclusion The SD - SimMIM method proposed in the paper effectively improves the analysis performance of dental panoramic radiographs through the self - distillation technique, especially in the case of limited data. Future work will further evaluate the application of this method in other downstream tasks, such as dental disease detection.

Enhanced Masked Image Modeling for Analysis of Dental Panoramic Radiographs

Self-Supervised Learning with Masked Image Modeling for Teeth Numbering, Detection of Dental Restorations, and Instance Segmentation in Dental Panoramic Radiographs

Masked Image Modeling Advances 3D Medical Image Analysis

Rethinking masked image modelling for medical image representation

A Deep Learning Framework with Pruning RoI Proposal for Dental Caries Detection in Panoramic X-ray Images.

DPML: Prior-Guided Multitask Learning for Dental Object Recognition on Limited Panoramic Radiograph Dataset

Collaborative deep learning model for tooth segmentation and identification using panoramic radiographs

Enhanced Diagnostic Accuracy for Dental Caries and Anomalies in Panoramic Radiographs Using a Custom Deep Learning Model

Deep Learning-Based Multiclass Instance Segmentation for Dental Lesion Detection

MedIM: Boost Medical Image Representation via Radiology Report-Guided Masking

Semi-supervised classification of dental conditions in panoramic radiographs using large language model and instance segmentation: A real-world dataset evaluation

SSAD: Self-supervised Auxiliary Detection Framework for Panoramic X-ray based Dental Disease Diagnosis

AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-guided Self-masking

Robust Automated Teeth Identification from Dental Radiographs Using Deep Learning.

HybridMIM: A Hybrid Masked Image Modeling Framework for 3D Medical Image Segmentation

Tufts Dental Database: A Multimodal Panoramic X-Ray Dataset for Benchmarking Diagnostic Systems

Automated Dental Image Analysis by Deep Learning on Small Dataset

Self-distilled Masked Attention guided masked image modeling with noise Regularized Teacher (SMART) for medical image analysis

Ai-Aided Diagnosis of Periodontitis in Oral X-Ray Images

Precise Dental Staging Method Through Panoramic Radiographs Based on Deep Learning

Hybrid Masked Image Modeling for 3D Medical Image Segmentation