Biomedical engineering--an annotated guide to sources of information. 1.

P. Kettlewell

Biomedical Engineering

Abstract:

What problem does this paper attempt to address?

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Jan Funke,Fabian Tschopp,William Grisaitis,Arlo Sheridan,Chandan Singh,Stephan Saalfeld,Srinivas C Turaga

DOI: https://doi.org/10.1109/TPAMI.2018.2835450

Abstract:We present a method combining affinity prediction with region agglomeration, which improves significantly upon the state of the art of neuron segmentation from electron microscopy (EM) in accuracy and scalability. Our method consists of a 3D U-Net, trained to predict affinities between voxels, followed by iterative region agglomeration. We train using a structured loss based on Malis, encouraging topologically correct segmentations obtained from affinity thresholding. Our extension consists of two parts: First, we present a quasi-linear method to compute the loss gradient, improving over the original quadratic algorithm. Second, we compute the gradient in two separate passes to avoid spurious gradient contributions in early training stages. Our predictions are accurate enough that simple learning-free percentile-based agglomeration outperforms more involved methods used earlier on inferior predictions. We present results on three diverse EM datasets, achieving relative improvements over previous results of 27, 15, and 250 percent. Our findings suggest that a single method can be applied to both nearly isotropic block-face EM data and anisotropic serial sectioned EM data. The runtime of our method scales linearly with the size of the volume and achieves a throughput of $\sim$∼ 2.6 seconds per megavoxel, qualifying our method for the processing of very large datasets.
Weighted average ensemble-based semantic segmentation in biological electron microscopy images

Kavitha Shaga Devan,Hans A. Kestler,Clarissa Read,Paul Walther

DOI: https://doi.org/10.1007/s00418-022-02148-3

2022-08-22

Histochemistry and Cell Biology

Abstract:Semantic segmentation of electron microscopy images using deep learning methods is a valuable tool for the detailed analysis of organelles and cell structures. However, these methods require a large amount of labeled ground truth data that is often unavailable. To address this limitation, we present a weighted average ensemble model that can automatically segment biological structures in electron microscopy images when trained with only a small dataset. Thus, we exploit the fact that a combination of diverse base-learners is able to outperform one single segmentation model. Our experiments with seven different biological electron microscopy datasets demonstrate quantitative and qualitative improvements. We show that the Grad-CAM method can be used to interpret and verify the prediction of our model. Compared with a standard U-Net, the performance of our method is superior for all tested datasets. Furthermore, our model leverages a limited number of labeled training data to segment the electron microscopy images and therefore has a high potential for automated biological applications.

cell biology,microscopy
Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images

James P. Horwath,Dmitri N. Zakharov,Rémi Mégret,Eric A. Stach

DOI: https://doi.org/10.1038/s41524-020-00363-x

IF: 12.256

2020-07-29

npj Computational Materials

Abstract:Abstract Cutting edge deep learning techniques allow for image segmentation with great speed and accuracy. However, application to problems in materials science is often difficult since these complex models may have difficultly learning meaningful image features that would enable extension to new datasets. In situ electron microscopy provides a clear platform for utilizing automated image analysis. In this work, we consider the case of studying coarsening dynamics in supported nanoparticles, which is important for understanding, for example, the degradation of industrial catalysts. By systematically studying dataset preparation, neural network architecture, and accuracy evaluation, we describe important considerations in applying deep learning to physical applications, where generalizable and convincing models are required. With a focus on unique challenges that arise in high-resolution images, we propose methods for optimizing performance of image segmentation using convolutional neural networks, critically examining the application of complex deep learning models in favor of motivating intentional process design.

materials science, multidisciplinary,chemistry, physical
Two-stage Generative Models of Simulating Training Data at the Voxel Level for Large-Scale Microscopy Bioimage Segmentation

Deli Wang,Ting Zhao,Nenggan Zheng,Zhefeng Gong

DOI: https://doi.org/10.24963/ijcai.2019/664

2019-01-01

Abstract:Bioimage Informatics is a growing area that aims to extract biological knowledge from microscope images of biomedical samples automatically. Its mission is vastly challenging, however, due to the complexity of diverse imaging modalities and big scales of multi-dimensional images. One major challenge is automatic image segmentation, an essential step towards high-level modeling and analysis. While progresses in deep learning have brought the goal of automation much closer to reality, creating training data for producing powerful neural networks is often laborious. To provide a shortcut for this costly step, we propose a novel two-stage generative model for simulating voxel level training data based on a specially designed objective function of preserving foreground labels. Using segmenting neurons from LM (Light Microscopy) image stacks as a testing example, we showed that segmentation networks trained by our synthetic data were able to produce satisfactory results. Unlike other simulation methods available in the field, our method can be easily extended to many other applications because it does not involve sophisticated cell models and imaging mechanisms.
Automated Cell Structure Extraction for 3D Electron Microscopy by Deep Learning

Jin Kousaka,Atsuko H. Iwane,Yuichi Togashi

2024-09-27

Abstract:Modeling the 3D structures of cells and tissues is crucial in biology. Sequential cross-sectional images from electron microscopy provide high-resolution intracellular structure information. The segmentation of complex cell structures remains a laborious manual task for experts, demanding time and effort. This bottleneck in analyzing biological images requires efficient and automated solutions. In this study, the deep learning-based automated segmentation of biological images was explored to enable accurate reconstruction of the 3D structures of cells and organelles. An analysis system for the cell images of {\em Cyanidioschyzon merolae}, a primitive unicellular red algae, was constructed. This system utilizes sequential cross-sectional images captured by a focused ion beam scanning electron microscope (FIB-SEM). A U-Net was adopted and pre-training was performed to identify and segment cell organelles from single-cell images. In addition, the segmentation anything model (SAM) and 3D watershed algorithm were employed to extract individual 3D images of each cell from large-scale microscope images containing numerous cells. Finally, the pre-trained U-Net was applied to segment each structure within these 3D images. Through this procedure, the creation of 3D cell models could be fully automated. The developed approach can be applied to other cell types, and a versatile analysis system will be built in future research. The adoption of other deep learning techniques and combinations of image processing methods will also be explored to enhance the segmentation accuracy further.

Quantitative Methods
Automatic Reconstruction of Mitochondria and Endoplasmic Reticulum in Electron Microscopy Volumes by Deep Learning

Jing Liu,Linlin Li,Yang Yang,Bei Hong,Xi Chen,Qiwei Xie,Hua Han

DOI: https://doi.org/10.3389/fnins.2020.00599

IF: 4.3

2020-07-21

Frontiers in Neuroscience

Abstract:Together, mitochondria and the endoplasmic reticulum (ER) occupy more than 20% of a cell's volume, and morphological abnormality may lead to cellular function disorders. With the rapid development of large-scale electron microscopy (EM), manual contouring and three-dimensional (3D) reconstruction of these organelles has previously been accomplished in biological studies. However, manual segmentation of mitochondria and ER from EM images is time consuming and thus unable to meet the demands of large data analysis. Here, we propose an automated pipeline for mitochondrial and ER reconstruction, including the mitochondrial and ER contact sites (MAMs). We propose a novel recurrent neural network to detect and segment mitochondria and a fully residual convolutional network to reconstruct the ER. Based on the sparse distribution of synapses, we use mitochondrial context information to rectify the local misleading results and obtain 3D mitochondrial reconstructions. The experimental results demonstrate that the proposed method achieves state-of-the-art performance.

neurosciences
Automated segmentation of cell organelles in volume electron microscopy using deep learning

Nebojša Nešić,Xavier Heiligenstein,Lydia Zopf,Valentin Blüml,Katharina S. Keuenhof,Michael Wagner,Johanna L. Höög,Heng Qi,Zhiyang Li,Georgios Tsaramirsis,Christopher J. Peddie,Miloš Stojmenović,Andreas Walter

DOI: https://doi.org/10.1002/jemt.24548

2024-03-20

Microscopy Research and Technique

Abstract:Our deep‐learning based segmentation workflow efficiently quantifies 3D subcellular structures in diverse electron microscopy modalities and cell lines, outperforming manual segmentation. It promises high‐throughput quantitative cell biology, ensuring rapid and accurate analysis for meaningful statistical insights. Recent advances in computing power triggered the use of artificial intelligence in image analysis in life sciences. To train these algorithms, a large enough set of certified labeled data is required. The trained neural network is then capable of producing accurate instance segmentation results that will then need to be re‐assembled into the original dataset: the entire process requires substantial expertise and time to achieve quantifiable results. To speed‐up the process, from cell organelle detection to quantification across electron microscopy modalities, we propose a deep‐learning based approach for fast automatic outline segmentation (FAMOUS), that involves organelle detection combined with image morphology, and 3D meshing to automatically segment, visualize and quantify cell organelles within volume electron microscopy datasets. From start to finish, FAMOUS provides full segmentation results within a week on previously unseen datasets. FAMOUS was showcased on a HeLa cell dataset acquired using a focused ion beam scanning electron microscope, and on yeast cells acquired by transmission electron tomography. Research Highlights Introducing a rapid, multimodal machine‐learning workflow for the automatic segmentation of 3D cell organelles. Successfully applied to a variety of volume electron microscopy datasets and cell lines. Outperforming manual segmentation methods in time and accuracy. Enabling high‐throughput quantitative cell biology.

biology,microscopy,anatomy & morphology
EM-stellar: benchmarking deep learning for electron microscopy image segmentation

Afshin Khadangi,Thomas Boudier,Vijay Rajagopal

DOI: https://doi.org/10.1101/2020.07.15.203836

2020-07-15

Abstract:Abstract The inherent low contrast of electron microscopy (EM) datasets presents a significant challenge for rapid segmentation of cellular ultrastructures from EM data. This challenge is particularly prominent when working with high resolution big-datasets that are now acquired using electron tomography and serial block-face imaging techniques. Deep learning (DL) methods offer an exciting opportunity to automate the segmentation process by learning from manual annotations of a small sample of EM data. While many DL methods are being rapidly adopted to segment EM data no benchmark analysis has been conducted on these methods to date. We present EM-stellar, a Jupyter Notebook platform that is hosted on google Colab that can be used to benchmark the performance of a range of state-of-the-art DL methods on user-provided datasets. Using EM-Stellar we show that the performance of any DL method is dependent on the properties of the images being segmented. It also follows that no single DL method performs consistently across all performance evaluation metrics.
Semi-Supervised Segmentation of Mitochondria from Electron Microscopy Images Using Spatial Continuity

Yunpeng Xiao,Youpeng Zhao,Ge Yang

DOI: https://doi.org/10.1109/ISBI52829.2022.9761519

2022-06-06

Abstract:Morphology of mitochondria plays critical roles in mediating their physiological functions. Accurate segmentation of mitochondria from 3D electron microscopy (EM) images is essential to quantitative characterization of their morphology at the nanometer scale. Fully supervised deep learning models developed for this task achieve excellent performance but require substantial amounts of annotated data for training. However, manual annotation of EM images is laborious and time-consuming because of their large volumes, limited contrast, and low signal-to-noise ratios (SNRs). To overcome this challenge, we propose a semi-supervised deep learning model that segments mitochondria by leveraging the spatial continuity of their structural, morphological, and contextual information in both labeled and unlabeled images. We use random piecewise affine transformation to synthesize comprehensive and realistic mitochondrial morphology for augmentation of training data. Experiments on the EPFL dataset show that our model achieves performance similar as that of state-of-the-art fully supervised models but requires only ~20% of their annotated training data. Our semi-supervised model is versatile and can also accurately segment other spatially continuous structures from EM images. Data and code of this study are openly accessible at <a class="link-external link-https" href="https://github.com/cbmi-group/MPP" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Machine Learning
Image Segmentation Using Deep Learning: A Survey

Shervin Minaee,Yuri Boykov,Fatih Porikli,Antonio Plaza,Nasser Kehtarnavaz,Demetri Terzopoulos

2020-11-15

Abstract:Image segmentation is a key topic in image processing and computer vision with applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among many others. Various algorithms for image segmentation have been developed in the literature. Recently, due to the success of deep learning models in a wide range of vision applications, there has been a substantial amount of works aimed at developing image segmentation approaches using deep learning models. In this survey, we provide a comprehensive review of the literature at the time of this writing, covering a broad spectrum of pioneering works for semantic and instance-level segmentation, including fully convolutional pixel-labeling networks, encoder-decoder architectures, multi-scale and pyramid based approaches, recurrent networks, visual attention models, and generative models in adversarial settings. We investigate the similarity, strengths and challenges of these deep learning models, examine the most widely used datasets, report performances, and discuss promising future research directions in this area.

Computer Vision and Pattern Recognition,Machine Learning
User-Accessible Machine Learning Approaches for Cell Segmentation and Analysis in Tissue

Seth Winfree

DOI: https://doi.org/10.3389/fphys.2022.833333

IF: 4

2022-03-10

Frontiers in Physiology

Abstract:Advanced image analysis with machine and deep learning has improved cell segmentation and classification for novel insights into biological mechanisms. These approaches have been used for the analysis of cells in situ , within tissue, and confirmed existing and uncovered new models of cellular microenvironments in human disease. This has been achieved by the development of both imaging modality specific and multimodal solutions for cellular segmentation, thus addressing the fundamental requirement for high quality and reproducible cell segmentation in images from immunofluorescence, immunohistochemistry and histological stains. The expansive landscape of cell types-from a variety of species, organs and cellular states-has required a concerted effort to build libraries of annotated cells for training data and novel solutions for leveraging annotations across imaging modalities and in some cases led to questioning the requirement for single cell demarcation all together. Unfortunately, bleeding-edge approaches are often confined to a few experts with the necessary domain knowledge. However, freely available, and open-source tools and libraries of trained machine learning models have been made accessible to researchers in the biomedical sciences as software pipelines, plugins for open-source and free desktop and web-based software solutions. The future holds exciting possibilities with expanding machine learning models for segmentation via the brute-force addition of new training data or the implementation of novel network architectures, the use of machine and deep learning in cell and neighborhood classification for uncovering cellular microenvironments, and the development of new strategies for the use of machine and deep learning in biomedical research.

physiology
Advancing electron microscopy using deep learning

K Chen,A S Barnard

DOI: https://doi.org/10.1088/2515-7639/ad229b

2024-02-08

Journal of Physics: Materials

Abstract:Abstract Electron microscopy, a sub-field of microanalysis, is critical to many fields of research. The widespread use of electron microscopy for imaging molecules and materials has had an enormous impact on our understanding of countless systems and has accelerated impacts in drug discovery and materials design, for electronic, energy, environment and health applications. With this success a bottleneck has emerged, as the rate at which we can collect data has significantly exceeded the rate at which we can analyze it. Fortunately, this has coincided with the rise of advanced computational methods, including data science and machine learning. Deep learning (DL), a sub-field of machine learning capable of learning from large quantities of data such as images, is ideally suited to overcome some of the challenges of electron microscopy at scale. There are a variety of different DL approaches relevant to the field, with unique advantages and disadvantages. In this review, we describe some well-established methods, with some recent examples, and introduce some new methods currently emerging in computer science. Our summary of DL is designed to guide electron microscopists to choose the right DL algorithm for their research and prepare for their digital future.

English Else
Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

Chi Xiao,Xi Chen,Weifu Li,Linlin Li,Lu Wang,Qiwei Xie,Hua Han

DOI: https://doi.org/10.3389/fnana.2018.00092

2018-01-01

Frontiers in Neuroanatomy

Abstract:Recent studies have supported the relation between mitochondrial functions and degenerative disorders related to ageing, such as Alzheimer's and Parkinson's diseases. Since these studies have exposed the need for detailed and high-resolution analysis of physical alterations in mitochondria, it is necessary to be able to perform segmentation and 3D reconstruction of mitochondria. However, due to the variety of mitochondrial structures, automated mitochondria segmentation and reconstruction in electron microscopy (EM) images have proven to be a difficult and challenging task. This paper puts forward an effective and automated pipeline based on deep learning to realize mitochondria segmentation in different EM images. The proposed pipeline consists of three parts: (1) utilizing image registration and histogram equalization as image pre-processing steps to maintain the consistency of the dataset; (2) proposing an effective approach for 3D mitochondria segmentation based on a volumetric, residual convolutional and deeply supervised network; and (3) employing a 3D connection method to obtain the relationship of mitochondria and displaying the 3D reconstruction results. To our knowledge, we are the first researchers to utilize a 3D fully residual convolutional network with a deeply supervised strategy to improve the accuracy of mitochondria segmentation. The experimental results on anisotropic and isotropic EM volumes demonstrate the effectiveness of our method, and the Jaccard index of our segmentation (91.8% in anisotropy, 90.0% in isotropy) and F1 score of detection (92.2% in anisotropy, 90.9% in isotropy) suggest that our approach achieved state-of-the-art results. Our fully automated pipeline contributes to the development of neuroscience by providing neurologists with a rapid approach for obtaining rich mitochondria statistics and helping them elucidate the mechanism and function of mitochondria.
Deep Generative Models-Assisted Automated Labeling for Electron Microscopy Images Segmentation

Wenhao Yuan,Bingqing Yao,Shengdong Tan,Fengqi You,Qian He

2024-07-29

Abstract:The rapid advancement of deep learning has facilitated the automated processing of electron microscopy (EM) big data stacks. However, designing a framework that eliminates manual labeling and adapts to domain gaps remains challenging. Current research remains entangled in the dilemma of pursuing complete automation while still requiring simulations or slight manual annotations. Here we demonstrate tandem generative adversarial network (tGAN), a fully label-free and simulation-free pipeline capable of generating EM images for computer vision training. The tGAN can assimilate key features from new data stacks, thus producing a tailored virtual dataset for the training of automated EM analysis tools. Using segmenting nanoparticles for analyzing size distribution of supported catalysts as the demonstration, our findings showcased that the recognition accuracy of tGAN even exceeds the manually-labeling method by 5%. It can also be adaptively deployed to various data domains without further manual manipulation, which is verified by transfer learning from HAADF-STEM to BF-TEM. This generalizability may enable it to extend its application to a broader range of imaging characterizations, liberating microscopists and materials scientists from tedious dataset annotations.

Materials Science,Image and Video Processing
Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Yan Xu,Rixiang Quan,Weiting Xu,Yi Huang,Xiaolong Chen,Fengyuan Liu

DOI: https://doi.org/10.3390/bioengineering11101034

2024-10-16

Abstract:Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.
Deep learning in mesoscale brain image analysis: A review

Runze Chen,Min Liu,Weixun Chen,Yaonan Wang,Erik Meijering

DOI: https://doi.org/10.1016/j.compbiomed.2023.107617

Abstract:Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.
Medical Image Segmentation Using Deep Learning: Review

Aida Hussein,Walaa Abd-Elhafiez,Elnomery Zanaty,Mohamed Hussein

DOI: https://doi.org/10.21608/aujst.2023.312707

2023-06-01

Aswan University Journal of Sciences and Technology

Abstract:Medical image segmentation has greatly increased healthcare sustainability. It is currently a major research area in the field of computer vision. The many artefacts inherent in medical images make it a difficult process to segment them. Deep neural models have recently demonstrated their use in a variety of image segmentation applications. With the rapid advancement of deep convolutional neural networks, medical image processing has become a study hotspot development of deep learning. The primary focus of this study is the deep learning-based segmentation of medical images.. This study gives an overview of the literature in the area of deep convolutional neural network-based medical image segmentation. The article looks at several popular medical image datasets, several segmentation task evaluation measures, and the effectiveness of various CNN-based networks. The current study also examines several issues in the area of segmenting medical images and various state-of-the-art solutions accessible in the literature, in contrast to the existing survey and review articles. The paper has several contributions which are as follows: Firstly, the present study provides an overview of the current state of the deep neural network structures utilized for medical image segmentation. Secondly, the paper describes the publicly available techniques of Network Training. Thirdly, it presents the various performance metrics employed for evaluating the deep learning segmentation models and the medical image segmentation datasets. Finally, the paper also gives an insight into the major challenges faced in the field of image segmentation and their state-of-the-art solutions
Deep learning based supervised semantic segmentation of Electron Cryo-Subtomograms

Chang Liu,Xiangrui Zeng,Ruogu Lin,Xiaodan Liang,Zachary Freyberg,Eric Xing,Min Xu

DOI: https://doi.org/10.48550/arXiv.1802.04087

2018-02-12

Abstract:Cellular Electron Cryo-Tomography (CECT) is a powerful imaging technique for the 3D visualization of cellular structure and organization at submolecular resolution. It enables analyzing the native structures of macromolecular complexes and their spatial organization inside single cells. However, due to the high degree of structural complexity and practical imaging limitations, systematic macromolecular structural recovery inside CECT images remains challenging. Particularly, the recovery of a macromolecule is likely to be biased by its neighbor structures due to the high molecular crowding. To reduce the bias, here we introduce a novel 3D convolutional neural network inspired by Fully Convolutional Network and Encoder-Decoder Architecture for the supervised segmentation of macromolecules of interest in subtomograms. The tests of our models on realistically simulated CECT data demonstrate that our new approach has significantly improved segmentation performance compared to our baseline approach. Also, we demonstrate that the proposed model has generalization ability to segment new structures that do not exist in training data.

Quantitative Methods,Computer Vision and Pattern Recognition,Machine Learning
A deep learning-based strategy for producing dense 3D segmentations from sparsely annotated 2D images

Vijay Venu Thiyagarajan,Arlo Sheridan,Kristen M. Harris,Uri Manor

DOI: https://doi.org/10.1101/2024.06.14.599135

2024-06-15

Abstract:Producing dense 3D reconstructions from biological imaging data is a challenging instance segmentation task that requires significant ground-truth training data for effective and accurate deep learning-based models. Generating training data requires intense human effort to annotate each instance of an object across serial section images. Our focus is on the especially complicated brain neuropil, comprising an extensive interdigitation of dendritic, axonal, and glial processes visualized through serial section electron microscopy. We developed a novel deep learning-based method to generate dense 3D segmentations rapidly from sparse 2D annotations of a few objects on single sections. Models trained on the rapidly generated segmentations achieved similar accuracy as those trained on expert dense ground-truth annotations. Human time to generate annotations was reduced by three orders of magnitude and could be produced by non-expert annotators. This capability will democratize generation of training data for large image volumes needed to achieve brain circuits and measures of circuit strengths.

Neuroscience
Uncertainty-aware particle segmentation for electron microscopy at varied length scales

Luca Rettenberger,Nathan J. Szymanski,Yan Zeng,Jan Schuetzke,Shilong Wang,Gerbrand Ceder,Markus Reischl

DOI: https://doi.org/10.1038/s41524-024-01302-w

IF: 12.256

2024-06-13

npj Computational Materials

Abstract:Electron microscopy is indispensable for examining the morphology and composition of solid materials at the sub-micron scale. To study the powder samples that are widely used in materials development, scanning electron microscopes (SEMs) are increasingly used at the laboratory scale to generate large datasets with hundreds of images. Parsing these images to identify distinct particles and determine their morphology requires careful analysis, and automating this process remains challenging. In this work, we enhance the Mask R-CNN architecture to develop a method for automated segmentation of particles in SEM images. We address several challenges inherent to measurements, such as image blur and particle agglomeration. Moreover, our method accounts for prediction uncertainty when such issues prevent accurate segmentation of a particle. Recognizing that disparate length scales are often present in large datasets, we use this framework to create two models that are separately trained to handle images obtained at low or high magnification. By testing these models on a variety of inorganic samples, our approach to particle segmentation surpasses an established automated segmentation method and yields comparable results to the predictions of three domain experts, revealing comparable accuracy while requiring a fraction of the time. These findings highlight the potential of deep learning in advancing autonomous workflows for materials characterization.

materials science, multidisciplinary,chemistry, physical

Biomedical engineering--an annotated guide to sources of information. 1.

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Weighted average ensemble-based semantic segmentation in biological electron microscopy images

Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images

Two-stage Generative Models of Simulating Training Data at the Voxel Level for Large-Scale Microscopy Bioimage Segmentation

Automated Cell Structure Extraction for 3D Electron Microscopy by Deep Learning

Automatic Reconstruction of Mitochondria and Endoplasmic Reticulum in Electron Microscopy Volumes by Deep Learning

Automated segmentation of cell organelles in volume electron microscopy using deep learning

EM-stellar: benchmarking deep learning for electron microscopy image segmentation

Semi-Supervised Segmentation of Mitochondria from Electron Microscopy Images Using Spatial Continuity

Image Segmentation Using Deep Learning: A Survey

User-Accessible Machine Learning Approaches for Cell Segmentation and Analysis in Tissue

Advancing electron microscopy using deep learning

Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

Deep Generative Models-Assisted Automated Labeling for Electron Microscopy Images Segmentation

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Deep learning in mesoscale brain image analysis: A review

Medical Image Segmentation Using Deep Learning: Review

Deep learning based supervised semantic segmentation of Electron Cryo-Subtomograms

A deep learning-based strategy for producing dense 3D segmentations from sparsely annotated 2D images

Uncertainty-aware particle segmentation for electron microscopy at varied length scales