Abstract:Background and objective: Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs. Methods: Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles. Results: We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations. Conclusions: We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.

Simulated HRTEM images of nanoparticles to train a neural network to classify nanoparticles for crystallinity

Deep learning-assisted analysis of HRTEM images of crystalline nanoparticles

Automated analysis of transmission electron micrographs of metallic nanoparticles by machine learning

Automated Classification of Nanoparticles with Various Ultrastructures and Sizes

Recognition of the three-dimensional structure of small metal nanoparticles by a supervised artificial neural network

Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis

Phase division and recognition of crystal HRTEM images based on machine learning and deep learning

Automatic Identification of Crystal Structures and Interfaces via Artificial-Intelligence-based Electron Microscopy

A robust synthetic data generation framework for machine learning in High-Resolution Transmission Electron Microscopy (HRTEM)

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images

Neural Network-based Classification of Crystal Symmetries from X-Ray Diffraction Patterns

FFT pattern recognition of crystal HRTEM image with deep learning

Automated Crystal System Identification from Electron Diffraction Patterns Using Multiview Opinion Fusion Machine Learning

A deep learning approach for determining the chiral indices of carbon nanotubes from high-resolution transmission electron microscopy images

High-Throughput, Algorithmic Determination of Nanoparticle Structure from Electron Microscopy Images

Deep learning for automated size and shape analysis of nanoparticles in scanning electron microscopy

Structural classification of Ag and Cu nanocrystals with machine learning

Microcrystal Growth Pathways Investigated with Machine Learning Segmentation and Classification in Scanning Electron Microscopy.

Paradigm shift in electron-based crystallography via machine learning

Learning-based Defect Recognition for Quasi-Periodic Microscope Images

Neural networks trained on synthetically generated crystals can extract structural information from ICSD powder X-ray diffractograms