Abstract:Background and objective: Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs. Methods: Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles. Results: We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations. Conclusions: We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.

Convolutional neural network for 2D-reconstructions of rough particles from their experimental speckle images in Interferometric Particle Imaging

Reconstruction of the shape of irregular rough particles from their interferometric images using a convolutional neural network

A Calibration-Informed Deep Learning Model for Three-Dimensional Particle Reconstruction of Volumetric Particle Image Velocimetry

Untrained Neural Network Enabling Fast and Universal Structured-Illumination Microscopy

Convolutional Neural Network Based Computational Imaging Spectroscopy

Particle Identification in Particle Tracking Velocimetry Using Two-Stage Neural Networks

Isotropic Resolution in Fluorescence Imaging by Single Particle Reconstruction

Reconstruction from Multiple Poses in Fluorescence Imaging: Proof of Concept

Reconstruction of nanoscale particles from single-shot wide-angle FEL diffractions patterns with physics-informed neural networks

Predicting 3D particles shapes based on 2D images by using convolutional neural network

Deep Learning for Position Fixing in the Micron Scale by Using Convolutional Neural Networks

Adaptive 3D convolutional neural network-based reconstruction method for 3D coherent diffraction imaging

Fast and robust single particle reconstruction in 3D fluorescence microscopy

A deep neural network architecture for reliable 3D position and size determination for Lagrangian particle tracking using a single camera

2D Convolutional Neural Network for Event Reconstruction in IceCube DeepCore

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images

Convolutional neural networks automate detection for tracking of submicron scale particles in 2D and 3D

Convolutional neural networks for grazing incidence x-ray scattering patterns: thin film structure identification

Colloidoscope: Detecting Dense Colloids in 3d with Deep Learning

Scalable Deep Convolutional Neural Networks for Sparse, Locally Dense Liquid Argon Time Projection Chamber Data

Physical model simulator-trained neural network for computational 3D phase imaging of multiple-scattering samples