Abstract:Background and objective: Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs. Methods: Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles. Results: We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations. Conclusions: We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.

Dataset of gold nanoparticle sizes and morphologies extracted from literature-mined microscopy images

Text-mined dataset of gold nanoparticle synthesis procedures, morphologies, and size entities

Data-driven analysis of text-mined seed-mediated growth AuNP syntheses

Leveraging Machine Learning for Size and Shape Analysis of Nanoparticles: A Shortcut to Electron Microscopy

A New Strategy for Microbial Taxonomic Identification through Micro-Biosynthetic Gold Nanoparticles and Machine Learning

Extracting Structured Seed-Mediated Gold Nanorod Growth Procedures from Literature with GPT-3

High-Throughput, Algorithmic Determination of Nanoparticle Structure from Electron Microscopy Images

Automated Classification of Nanoparticles with Various Ultrastructures and Sizes

Comparative Characterisation of Non-Monodisperse Gold Nanoparticle Populations by X-ray Scattering and Electron Microscopy

AuNR-SMA: Automated Gold Nanorod Spectral Morphology Analysis Pipeline

Morphology of gold nanoparticles determined by full-curve fitting of the light absorption spectrum. Comparison with X-ray scattering and electron microscopy data

Techniques for Accurate Sizing of Gold Nanoparticles Using Dynamic Light Scattering with Particular Application to Chemical and Biological Sensing Based on Aggregate Formation.

Size distributions of gold nanoparticles in solution measured by single-particle mass photometry

Statistically Representative Metrology of Nanoparticles via Unsupervised Machine Learning of TEM Images

Bayesian object classification of gold nanoparticles

Resolving non-equilibrium shape variations amongst millions of gold nanoparticles

A label free quantification of gold nanoparticle at the single cell level using multi-column convolutional neural network (MC-CNN)

Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images

Recursive Detection and Analysis of Nanoparticles in Scanning Electron Microscopy Images

Simulated HRTEM images of nanoparticles to train a neural network to classify nanoparticles for crystallinity