Abstract:Objective: Digital microscopy is used to monitor particulates such as protein aggregates within biopharmaceutical products. The images that result encode a wealth of information that is underutilized in pharmaceutical process monitoring. For example, images of particles in protein drug products typically are analyzed only to obtain particle counts and size distributions, even though the images also reflect particle characteristics such as shape and refractive index. Multiple groups have demonstrated that convolutional neural networks (CNNs) can extract information from images of protein aggregates allowing assignment of the likely stress at the "root-cause" of aggregation. A practical limitation of previous CNN-based approaches is that the potential aggregation-inducing stresses must be known a priori, disallowing identification of particles produced by unknown stresses. Methods: We demonstrate an expanded CNN analysis of flow imaging microscopy (FIM) images incorporating judiciously chosen particle standards within a recently proposed "fingerprinting algorithm" (Biotechnol. & Bioeng. (2020) 117:3322) that allows detection of particles formed by unknown root-causes. We focus on ethylene tetrafluoroethylene (ETFE) microparticles as standard surrogates for protein aggregates. We quantify the sensitivity of the new algorithm to experimental parameters such as microscope focus and solution refractive index changes, and explore how FIM sample noise affects statistical testing procedures. Results & conclusions: Applied to real-world microscopy images of protein aggregates, the algorithm reproducibly detects complex, distinguishing "textural features" of particles that are not easily described by standard morphological measurements. This offers promise for quality control applications and for detecting shifts in protein aggregate populations due to stresses resulting from unknown process upsets.

Using Deep Convolutional Neural Networks to Circumvent Morphological Feature Specification when Classifying Subvisible Protein Aggregates from Micro-Flow Images

Testing Precision Limits of Neural Network-Based Quality Control Metrics in High-Throughput Digital Microscopy

Novel machine learning models for flow imaging microscopy sub-visible particle classification in protein formulations

Features in Backgrounds of Microscopy Images Introduce Biases in Machine Learning Analyses

Learning Consistent Subcellular Landmarks to Quantify Changes in Multiplexed Protein Maps.

Deep localization of protein structures in fluorescence microscopy images

Classifying Mixed Patterns of Proteins in High-Throughput Microscopy Images Using Deep Neural Networks

Transfer Learning Analysis for Subvisible Particle Flow Imaging of Pharmaceutical Formulations

DeepLearnMOR: a Deep-Learning Framework for Fluorescence Image-Based Classification of Organelle Morphology.

Stain-free approach to determine and monitor cell heath using supervised and unsupervised image-based deep learning

Deep learning approach to Fourier ptychographic microscopy

Multi-labelled Proteins Recognition for High-Throughput Microscopy Images Using Deep Convolutional Neural Networks.

Image‐based characterization of flocculation processes through PLS inspired representation learning in convolutional neural networks

Automatic labeling of molecular biomarkers of whole slide immunohistochemistry images using fully convolutional networks

Multimodal registration of FISH and nanoSIMS images using convolutional neural network models

Protein Subcellular Localization Prediction by Concatenation of Convolutional Blocks for Deep Features Extraction From Microscopic Images

Classification of X-Ray Protein Crystallization Using Deep Convolutional Neural Networks with a Finder Module

Segmentation of histological images and fibrosis identification with a convolutional neural network

A Multi-Scale Multi-Model Deep Neural Network Via Ensemble Strategy on High-Throughput Microscopy Image for Protein Subcellular Localization

A deep learning-based approach to extraction of filler morphology in SEM images with the application of automated quality inspection

Efficient Cell Classification of Mitochondrial Images by Using Deep Learning