Abstract:Background and objective: Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs. Methods: Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles. Results: We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations. Conclusions: We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.

Deep Neutral Network-Based Electron Microscopy Image Recognition for Source Distinguishing of Anthropogenic and Natural Magnetic Particles

Deep-Learning Recognition of Scanning Transmission Electron Microscopy: Quantifying and Mitigating the Influence of Gaussian Noises

MagNet: Automated Magnetic Mineral Grain Morphometry Using Convolutional Neural Network

Deep-learning-based prediction of nanoparticle phase transitions during in situ transmission electron microscopy

A small-dataset-trained deep learning framework for identifying atoms on transmission electron microscopy images

Microscopic image recognition of diatoms based on deep learning.

Deep learning assisted high resolution microscopy image processing for phase segmentation in functional composite materials

Automated analysis of transmission electron micrographs of metallic nanoparticles by machine learning

Uncertainty-aware particle segmentation for electron microscopy at varied length scales

Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images

Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images

Using Computer Vision and Deep Learning for Nanoparticle Recognition on Scanning Probe Microscopy Images: Modified U-net Approach

Developing and Evaluating Deep Neural Network-Based Denoising for Nanoparticle TEM Images with Ultra-Low Signal-to-Noise

Machine Learning-Enabled Image Classification for Automated Electron Microscopy

Enhanced Nanoparticle Recognition via Deep Learning-Accelerated Plasmonic Sensing

Semi-supervised machine learning model for analysis of nanowire morphologies from transmission electron microscopy images

Machine Learning to Reveal Nanoparticle Dynamics from Liquid-Phase TEM Videos

Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase Contrast Transmission Electron Microscopy Images

Mie Sensing with Neural Networks: Recognition of Nano-Object Parameters, the Invisibility Point, and Restricted Models