Abstract:Abstract Our knowledge of cancer biology has advanced with the characterization of distinct cell types and cell states within heterogeneous tumor environments. Many methods for imaging and sorting tumor cells require biomarker labels that alter cell characteristics and create a selection bias. The use of label-free single-cell methods should further improve cancer studies involving viable, unperturbed cells for downstream assays such as RNA-Seq, cell culture expansion, and functional assays. To eliminate biomarkers and enable broader assessment of cells, we developed REM-I, a platform that characterizes and sorts unlabeled single cells based on high-dimensional morphology. Cells are captured with brightfield imaging and processed in real-time by self-supervised deep learning models to generate quantitative AI embeddings representative of cell morphology. A significant technical challenge in building REM-I was developing an AI model based on extracted features from cell images without prior knowledge of cell types, cell preparation, or other application-specific knowledge. Accordingly, we developed the Human Foundation Model (HFM), a hybrid architecture that combines self-supervised learning (SSL) and morphometrics (computer vision) to extract 115 dimensional embeddings representing cell morphology from high-resolution REM-I cell images. SSL produces a foundation model with high generalization capacity that enables hypothesis-free sample exploration and efficient generation of application-specific models. Meanwhile, computer vision extracts features that represent measurable and interpretable concepts (e.g., cell size, shape, texture, intensity). The training process for the HFM self-supervised backbone model utilizes the discriminatory power of supervised tasks. Using synthetic cells and three cancer cell lines, we trained and then validated the reproducibility and generalization capabilities of the resulting model. Results show combining deep learning and morphometrics improve interpretability of data and enable rapid characterization and classification of tumor cells with high accuracy. We also report on the Deepcell® Axon data suite, a tool to analyze data and customize reusable workflows. This includes the ability to store and manage data, visualize high dimensional data as low-dimensional projections, and train classifiers to identify and sort cell populations. To enable compatibility with user-preferred downstream analysis pipelines, Axon provides data export options for images, plots, and embeddings. Our approach allows users of all skill levels to access and interpret AI-enabled morphological profiling. Applications of REM-I include hypothesis-free evaluation of heterogeneous tumor samples, label-free cancer cell enrichment, characterization of distinct cell states, and multi-omic integration. Citation Format: Kiran Saini, Senzeyu Zhang, Ryan Carelli, Kevin B. Jacobs, Amy Wong-Thai, Cris Luengo, Vivian Lu, Anastasia Mavropoulos, Andreja Jovic, Jeanette Mei, Thomas Vollbrecht, Stephane C. Boutet, Mahyar Salek, Maddison Masaeli. Self-supervised foundation model captures high-dimensional morphology data from single cell brightfield images [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3523.

Self-supervised deep learning for highly efficient spatial immunophenotyping

Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images

Spatial Immunophenotyping from Whole-Slide Multiplexed Tissue Imaging Using Convolutional Neural Networks

Cell segmentation for immunofluorescence multiplexed images using two-stage domain adaptation and weakly labeled data for pre-training

Addressing Sparse Annotation: a Novel Semantic Energy Loss for Tumor Cell Detection from Histopathologic Images.

Hoechst Is All You Need: Lymphocyte Classification with Deep Learning

Abstract 3523: Self-supervised foundation model captures high-dimensional morphology data from single cell brightfield images

Deep Learning-Inferred Multiplex ImmunoFluorescence for IHC Image Quantification

Abstract 892: Pan-cancer tumor microenvironment profiling with multiplexed immunofluorescence microscopy and self-supervised learning

An AI-Ready Multiplex Staining Dataset for Reproducible and Accurate Characterization of Tumor Immune Microenvironment

Cytometry Masked Autoencoder: An Accurate and Interpretable Automated Immunophenotyper

Deep-Learning-Based Approach in Cancer-Region Assessment from HER2-SISH Breast Histopathology Whole Slide Images

Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Immunohistochemistry annotations enhance AI identification of lymphocytes and neutrophils in digitized H&E slides from inflammatory bowel disease

Automated Phenotyping via Cell Auto Training (CAT) on the Cell DIVE Platform

Semi-supervised variational autoencoder for cell feature extraction in multiplexed immunofluorescence images

Multiplex Immunohistochemical Analysis of the Spatial Immune Cell Landscape of the Tumor Microenvironment

Identification of cell types in multiplexed in situ images by combining protein expression and spatial information using CELESTA

Label-free 3-D quantitative phase imaging cytometry with deep learning: identifying naive, memory, and senescent T cells

Leveraging immuno-fluorescence data to reduce pathologist annotation requirements in lung tumor segmentation using deep learning