Abstract:Crystalline defects, such as line-like dislocations, play an important role for the performance and reliability of many metallic devices. Their interaction and evolution still poses a multitude of open questions to materials science and materials physics. In-situ TEM experiments can provide important insights into how dislocations behave and move. During such experiments, the dislocation microstructure is captured in form of videos. The analysis of individual video frames can provide useful insights but is limited by the capabilities of automated identification, digitization, and quantitative extraction of the dislocations as curved objects. The vast amount of data also makes manual annotation very time consuming, thereby limiting the use of Deep Learning-based, automated image analysis and segmentation of the dislocation microstructure. In this work, a parametric model for generating synthetic training data for segmentation of dislocations is developed. Even though domain scientists might dismiss synthetic training images sometimes as too artificial, our findings show that they can result in superior performance, particularly regarding the generalizing of the Deep Learning models with respect to different microstructures and imaging conditions. Additionally, we propose an enhanced deep learning method optimized for segmenting overlapping or intersecting dislocation lines. Upon testing this framework on four distinct real datasets, we find that our synthetic training data are able to yield high-quality results also on real images-even more so if fine-tune on a few real images was done.

What problem does this paper attempt to address?

This paper mainly discusses how to solve the problem of insufficient training data in the study of crystal defects such as dislocations. Researchers have developed a parameterized model to generate synthetic training data for dislocation segmentation. Although synthetic images may sometimes be considered artificial by domain scientists, studies have shown that this approach can improve the performance of deep learning models, especially in terms of generalization ability for different microstructures and imaging conditions. The paper points out that although traditional transmission electron microscopy (TEM) experiments can provide insights into dislocation behavior, the limitations of analyzing individual video frames lie in the limited ability to automatically identify, digitize, and extract dislocations as curve objects. The large amount of data also makes manual annotation time-consuming, limiting the use of deep learning-based automatic image analysis and dislocation microstructure segmentation. The authors propose an enhanced deep learning approach that optimizes the segmentation of overlapping or intersecting dislocation lines and test this framework on four different real datasets. It was found that synthetic training data can also produce high-quality results on real images, especially after fine-tuning with a small amount of real images. This approach demonstrates the potential of synthetic data in overcoming the limitations of manual annotation in TEM and opens up avenues for more efficient and accurate analysis of dislocation microstructures. Moreover, segmenting such thin and curved structures is a common task in many fields, so this method may also be applicable to other applications. Overall, the paper aims to improve the efficiency and accuracy of dislocation analysis in materials science by using synthetic data to address the problem of insufficient training data.

Deep Learning of Crystalline Defects from TEM images: A Solution for the Problem of "Never Enough Training Data"

Instance Segmentation of Dislocations in TEM Images

A Deep Learning Based Automatic Defect Analysis Framework for In-situ TEM Ion Irradiations

Learning-based Defect Recognition for Quasi-Periodic Microscope Images

Single-image based deep learning for precise atomic defects identification

Accurate quantification of dislocation loops in complex functional alloys enabled by deep learning image analysis

Single-Image-Based Deep Learning for Precise Atomic Defect Identification

A Deep Learning Based Dislocation Detection Method for Cylindrical Crystal Growth Process

Deep Learning on Atomistic Physical Fields of Graphene for Strain and Defect Engineering

AI-Guided Defect Detection Techniques to Model Single Crystal Diamond Growth

Bridging Theory with Experiment: Digital Twins and Deep Learning Segmentation of Defects in Monolayer MX2 Phases

Interactive Defect Segmentation in X-Ray Images Based on Deep Learning

Performance, Successes and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs

Digital twins and deep learning segmentation of defects in monolayer MX2 phases

Deep Learning Enabled Strain Mapping of Single-Atom Defects in 2D Transition Metal Dichalcogenides with Sub-picometer Precision

A convolutional neural network for defect classification in Bragg coherent X-ray diffraction

Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patterns

Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

Towards End-to-End Structure Solutions from Information-Compromised Diffraction Data via Generative Deep Learning

Nonlocal Machine Learning of Micro-Structural Defect Evolutions in Crystalline Materials