Uncertainty-aware particle segmentation for electron microscopy at varied length scales

Luca Rettenberger,Nathan J. Szymanski,Yan Zeng,Jan Schuetzke,Shilong Wang,Gerbrand Ceder,Markus Reischl

DOI: https://doi.org/10.1038/s41524-024-01302-w

IF: 12.256

2024-06-13

npj Computational Materials

Abstract:Electron microscopy is indispensable for examining the morphology and composition of solid materials at the sub-micron scale. To study the powder samples that are widely used in materials development, scanning electron microscopes (SEMs) are increasingly used at the laboratory scale to generate large datasets with hundreds of images. Parsing these images to identify distinct particles and determine their morphology requires careful analysis, and automating this process remains challenging. In this work, we enhance the Mask R-CNN architecture to develop a method for automated segmentation of particles in SEM images. We address several challenges inherent to measurements, such as image blur and particle agglomeration. Moreover, our method accounts for prediction uncertainty when such issues prevent accurate segmentation of a particle. Recognizing that disparate length scales are often present in large datasets, we use this framework to create two models that are separately trained to handle images obtained at low or high magnification. By testing these models on a variety of inorganic samples, our approach to particle segmentation surpasses an established automated segmentation method and yields comparable results to the predictions of three domain experts, revealing comparable accuracy while requiring a fraction of the time. These findings highlight the potential of deep learning in advancing autonomous workflows for materials characterization.

materials science, multidisciplinary,chemistry, physical

What problem does this paper attempt to address?

The problems that this paper attempts to solve are the challenges encountered in particle segmentation using scanning electron microscope (SEM) images at different length scales. Specifically, these problems include:

1. **Image Blurring**: Blurring that may occur during the scanning electron microscope imaging process results in unclear particle boundaries, making accurate segmentation difficult.
2. **Particle Agglomeration**: In powder samples, particles often agglomerate together, forming overlapping structures, which makes it difficult to identify individual particles.
3. **Image Differences at Different Magnifications**: There are significant differences in features between images at low and high magnifications, and they need to be processed separately.
4. **Prediction Uncertainty**: In some cases, due to the existence of the above problems, the model may produce inaccurate segmentation results. Therefore, it is very important to evaluate the confidence of the model for each particle segmentation.

To solve these problems, the author proposes an enhanced Mask R - CNN architecture that can:

- **Process Images at Different Magnifications**: By training two models separately to process images at low and high magnifications.
- **Estimate Prediction Uncertainty**: An uncertainty head is introduced to generate a confidence score for each segmented particle.
- **Improve Segmentation Accuracy**: By comparing with the traditional U - Net architecture, the superior performance of the enhanced Mask R - CNN in the particle segmentation task is verified.

These improvements give this method significant advantages in handling complex images and improving the efficiency of automated analysis.