Abstract:In many real-world scientific problems, generating ground truth (GT) for supervised learning is almost impossible. The causes include limitations imposed by scientific instrument, physical phenomenon itself, or the complexity of modeling. Performing artificial intelligence (AI) tasks such as segmentation, tracking, and analytics of small sub-cellular structures such as mitochondria in microscopy videos of living cells is a prime example. The 3D blurring function of microscope, digital resolution from pixel size, optical resolution due to the character of light, noise characteristics, and complex 3D deformable shapes of mitochondria, all contribute to making this problem GT hard. Manual segmentation of 100s of mitochondria across 1000s of frames and then across many such videos is not only herculean but also physically inaccurate because of the instrument and phenomena imposed limitations. Unsupervised learning produces less than optimal results and accuracy is important if inferences relevant to therapy are to be derived. In order to solve this unsurmountable problem, we bring modeling and deep learning to a nexus. We show that accurate physics based modeling of microscopy data including all its limitations can be the solution for generating simulated training datasets for supervised learning. We show here that our simulation-supervised segmentation approach is a great enabler for studying mitochondrial states and behaviour in heart muscle cells, where mitochondria have a significant role to play in the health of the cells. We report unprecedented mean IoU score of 91% for binary segmentation (19% better than the best performing unsupervised approach) of mitochondria in actual microscopy videos of living cells. We further demonstrate the possibility of performing multi-class classification, tracking, and morphology associated analytics at the scale of individual mitochondrion.

2D Label-free Prediction of Multiple Organelles Across Different Transmitted-light Microscopy Images with Bag-of-Experts

High-quality Label-Free Prediction of Fluorescence Images Through Stimulated Raman Scattering Imaging and Recurrent Deep Neural Network.

Predicting fluorescent labels in label-free microscopy images with pix2pix and adaptive loss in Light My Cells challenge

An Artificial Intelligence-Based Stacked Ensemble Approach for Prediction of Protein Subcellular Localization in Confocal Microscopy Images

Label-free prediction of cell painting from brightfield images

Multi-omics Prediction from High-content Cellular Imaging with Deep Learning

Label-free cell cycle analysis for high-throughput imaging flow cytometry

Cell-context dependent in silico organelle localization in label-free microscopy images

Improving quantitative prediction of protein subcellular locations in fluorescence images through deep generative models

High-Resolution In Silico Painting with Generative Models

Learning Consistent Subcellular Landmarks to Quantify Changes in Multiplexed Protein Maps.

Label-free prediction of fluorescence markers in bovine satellite cells using deep learning

Simulation-supervised deep learning for analysing organelles states and behaviour in living cells

In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images

Multilabel Learning for Protein Subcellular Location Prediction

Robust virtual staining of landmark organelles

Deep learning-based segmentation of subcellular organelles in high-resolution phase-contrast images

Protein Subcellular Localization Prediction by Concatenation of Convolutional Blocks for Deep Features Extraction From Microscopic Images

Imbalanced multi-modal multi-label learning for subcellular localization prediction of human proteins with both single and multiple sites

Artificial Intelligence for Label-free Cells Classification in Holographic Microscopy

Fluo-Fluo translation based on deep learning