Abstract:Many methods for imaging and sorting tumor cells require biomarker labels that perturb cells and may create a selection bias. The Deepcell platform can characterize and sort cells using only brightfield images of unlabeled single cells, thereby enabling more comprehensive and unbiased assessment of tumor cell morphology and heterogeneity. Cells are imaged with a high-speed camera during microfluidic flow and brightfield images are analyzed in real-time by deep learning models to generate quantitative embeddings, which are reproducible and multi-dimensional descriptions of cell morphology. One of the key technical challenges in building this platform was the development of an AI model to extract features from cell images from diverse human cells without prior knowledge of specific cell types, cell preparation, or other application-specific knowledge for an exploratory approach. Deepcell's 'Human Foundation Model' (HFM) is a feature encoder that transforms cell images into 128 dimensional embedding vectors. The model backbone, responsible for extracting image features, is based on the ResNet18 convolutional deep neural network architecture. Training utilizes a multi-task semi-supervised training framework that combines the VicReg self-supervised learning model, which learns images features without labels, along with supervised auxiliary models using labeled data. These sub-models enable the backbone model to recognize specific cell attributes such as granulation, pigmentation, characteristics of malignancy, attributes related to cellular states like apoptosis and necrosis. We augment the deep learning embeddings with computer vision derived cellular features, such as area, perimeter, intensity, and texture features to improve model interpretability and accuracy. Here, we describe how the HFM self-supervised backbone model was trained, the discriminatory power added by the supervised tasks, and validation of the reproducibility and generalization capabilities of the resulting model. We also demonstrate how the resulting embeddings can be visualized using the Deepcell Cloud software and how clusters of cells in our embedding space can be related to cell images, interpretable morphological features, and actionable biological characteristics. Potential applications of the Deepcell platform are diverse and promising, such as label-free enrichment of malignant cells and discovery of morphologically distinct subpopulations of heterogeneous cancer cell samples. Citation Format: Kevin B. Jacobs, Kiran Saini, Nianzhen Li, Senzeyu Zhang, Ryan Carelli, Amy Wong-Thai, Stephane C. Boutet, Anastasia Mavropoulos, Andreja Jovic, Jeanette Mei, Mahyar Salek, Maddison Masaeli. Deep learning models capture high dimensional features for cell morphology analysis. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited s); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl) nr 5361.

Deep learning of cell spatial organizations identifies clinically relevant insights in tissue images

Context aware decision support in neurosurgical oncology based on an efficient classification of endomicroscopic data

Graph deep learning for the characterization of tumour microenvironments from spatial protein profiles in tissue specimens

A Deep Learning Approach for Tissue Spatial Quantification and Genomic Correlations of Histopathological Images

GTADC: A Graph-Based Method for Inferring Cell Spatial Distribution in Cancer Tissues

Computational staining of pathology images to study tumor microenvironment in lung cancer

Abstract 5361: Deep learning models capture high dimensional features for cell morphology analysis

DeepCG: A cell graph model for predicting prognosis in lung adenocarcinoma

Predicting Gene Spatial Expression and Cancer Prognosis: An Integrated Graph and Image Deep Learning Approach Based on HE Slides

Deep Learning-Based 3D Single-Cell Imaging Analysis Pipeline Enables Quantification of Cell-Cell Interaction Dynamics in the Tumor Microenvironment

CCF-GNN: A Unified Model Aggregating Appearance, Microenvironment, and Topology for Pathology Image Classification

Interpretable spatial cell learning enhances the characterization of patient tissue microenvironments with highly multiplexed imaging data

Data from Deep Learning–Based 3D Single-Cell Imaging Analysis Pipeline Enables Quantification of Cell–Cell Interaction Dynamics in the Tumor Microenvironment

Deep learning-derived spatial organization features on histology images predicts prognosis in colorectal liver metastasis patients after hepatectomy

PathoGraph: An Attention-Based Graph Neural Network Capable of Prognostication Based on CD276 Labelling of Malignant Glioma Cells

A novel deep learning-based 3D cell segmentation framework for future image-based disease detection

Computational Staining of Pathology Images to Study the Tumor Microenvironment in Lung Cancer

Identification of Prognostic Spatial Organization Features in Colorectal Cancer Microenvironment Using Deep Learning on Histopathology Images

Deep Mining of Subtle Differences in Cell Morphology Via Deep Learning

Deep Learning-Assisted Assessing of Single Circulating Tumor Cell Viability via Cellular Morphology