Abstract:Background and objective: Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs. Methods: Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles. Results: We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations. Conclusions: We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.

Fully automated primary particle size analysis of agglomerates on transmission electron microscopy images via artificial neural networks

Image-Based Size Analysis of Agglomerated and Partially Sintered Particles via Convolutional Neural Networks

Workflow towards automated segmentation of agglomerated, non-spherical particles from electron microscopy images using artificial neural networks

Deep learning for automated size and shape analysis of nanoparticles in scanning electron microscopy

Automated Classification of Nanoparticles with Various Ultrastructures and Sizes

A deep learning solution for particle size analysis in low resolution inline microscopy images based on generative adversarial network

Generative adversarial networks assisted machine learning based automated quantification of grain size from scanning electron microscope back scatter images

Automated analysis of transmission electron micrographs of metallic nanoparticles by machine learning

Automatic Detection of Particle Size Distribution by Image Analysis Based on Local Adaptive Canny Edge Detection and Modified Circular Hough Transform.

Leveraging Machine Learning for Size and Shape Analysis of Nanoparticles: A Shortcut to Electron Microscopy

Reconstruction of nanoscale particles from single-shot wide-angle FEL diffractions patterns with physics-informed neural networks

Automated Grain Boundary Detection for Bright-Field Transmission Electron Microscopy Images via U-Net

Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

Deep learning-based size prediction for optical trapped nanoparticles and extracellular vesicles from limited bandwidth camera detection

Testing Precision Limits of Neural Network-Based Quality Control Metrics in High-Throughput Digital Microscopy

Simulated HRTEM images of nanoparticles to train a neural network to classify nanoparticles for crystallinity

Image processing and neural network technique for size characterization of gravel particles

Precise Analysis of Nanoparticle Size Distribution in TEM Image

Machine Learning Pipeline for Segmentation and Defect Identification from High Resolution Transmission Electron Microscopy Data

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images

A new method for high-resolution particle measurement with a large field of view via dual-view shadowgraph imaging