Abstract:Background and objective: Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs. Methods: Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles. Results: We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations. Conclusions: We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.

Deep Learning-Assisted Localisation of Nanoparticles in synthetically generated two-photon microscopy images

Particle Identification in Particle Tracking Velocimetry Using Two-Stage Neural Networks

Deep learning-based spectroscopic single-molecule localization microscopy

Enhanced Nanoparticle Recognition via Deep Learning-Accelerated Plasmonic Sensing

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images

Deep learning-driven adaptive optics for single-molecule localization microscopy

Model-free machine learning-based 3D single molecule localisation microscopy

Synthetic Image Rendering Solves Annotation Problem in Deep Learning Nanoparticle Segmentation

3D Single Molecule Localization Microscopy Via Deep Learning

Localization of Ultra-dense Emitters with Neural Networks

Neural network‐assisted localization of clustered point spread functions in single‐molecule localization microscopy

Deep Learning for Tracking of Intracellular Vesicles in Time-lapse Microscopy Images

A deep learning solution for particle size analysis in low resolution inline microscopy images based on generative adversarial network

Robust deep learning optical autofocus system applied to automated multiwell plate single molecule localization microscopy

Performance of deep learning restoration methods for the extraction of particle dynamics in noisy microscopy image sequences

Unsupervised deep learning enables blur-free super-resolution in two-photon microscopy

Learning an optimal PSF-pair for ultra-dense 3D localization microscopy

Multicolor localization microscopy by deep learning

Depth-Resolved Localization Microangiography in the NIR-II Window

Label-free multiplexed microtomography of endogenous subcellular dynamics using generalizable deep learning

Deep mouse brain two-photon near-infrared fluorescence imaging using a superconducting nanowire single-photon detector array