Abstract:Abstract Tissue-based sampling and diagnosis is the extraction of information from specifically limited spaces and its diagnostic significance of a certain object. Pathologists deal with issues related to tumor heterogeneity since analyzing a single sample does not necessarily capture a representative depiction of cancer, and a tissue biopsy usually only presents a small fraction of the tumor. Many multiplex tissue imaging platforms (MTIs) make the assumption that tissue microarrays (TMAs) containing small core samples of 2-dimensional (2D) tissue sections are a good approximation of bulk tumors although tumors are not 2D. However, emerging whole slide imaging (WSI) or 3D tumor atlases that employ MTIs like cyclic immunofluorescence (CyCIF) strongly challenge this assumption. In spite of the additional insight gathered by measuring the tumor microenvironment in WSI or 3D, it can be prohibitively expensive and time-consuming to process tens or hundreds of tissue sections with CyCIF. Even when resources are not limited, the criteria for region-of-interest (ROI) selection in tissues for downstream analysis remain largely qualitative and subjective as stratified sampling requires the knowledge of objects and evaluates their features. Although TMAs fail to adequately approximate whole tissue features, a theoretical subsampling of tissue exists that can best represent the tumor in the whole slide image. To address these challenges, we propose deep learning approaches to learn multi-modal image translation tasks from two aspects: 1) generative modeling approach to reconstruct 3D CyCIF representation and 2) co-embedding CyCIF image and Hematoxylin and Eosin (H&E) section to learn multi-mappings by a cross-domain translation for minimum representative ROI selection. We demonstrate that generative modeling enables a 3D virtual CyCIF reconstruction of a colorectal cancer specimen given a small subset of the imaging data at training time. By co-embedding histology and MTI features, we propose a simple convex optimization for objective ROI selection. We demonstrate the potential application of ROI selection and the efficiency of its performance with respect to cellular heterogeneity. The key takeaway is that this pipeline allows for intelligent representation from H&E images, which enables a plethora of subsequent analyses on this representation space with other multiplexed imaging platforms such as Multiplexed ion beam imaging (MIBI), Imaging Mass Cytometry (IMC) or NanoString GeoMX as only a few ROIs could be selected and analyzed using these platforms. Citation Format: Erik Burlingame, Luke Ternes, Eun Na Kim, Joe W. Gray, Young Hwan Chang. 3D multiplexed tissue imaging reconstruction and optimized region-of-interest (ROI) selection through deep learning model of channels embedding. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5431.

Revealing 3D cancer tissue structures using holotomography and virtual hematoxylin and eosin staining via deep learning

Fast and label-free 3D virtual H&E histology via active modulation-assisted dynamic full-field OCT

Abstract 7401: High-level multiplexed virtual immunofluorescence staining using holotomography and deep learning

Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual H&E staining

Unstained Tissue Imaging and Virtual Hematoxylin and Eosin Staining of Histologic Whole Slide Images

Open-top Bessel beam two-photon light sheet microscopy for three-dimensional pathology

3D multiplexed tissue imaging reconstruction and optimized region of interest (ROI) selection through deep learning model of channels embedding

Deep learning for rapid virtual H&E staining of label-free glioma tissue from hyperspectral images

Noninvasive virtual biopsy using micro-registered optical coherence tomography (OCT) in human subjects

Deep Learning Enables Rapid Whole-Organ Histological Imaging with Ultraviolet-Excited Sectioning Tomography

Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue

Ultrafast 3D Histological Imaging Based on a Minutes-Time Scale Tissue Clearing and Multidirectional Selective Plane Illumination Microscopy

Evaluation of tumor budding with virtual panCK stains generated by novel multi-model CNN framework

Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

Autofluorescence Virtual Staining System for H&E Histology and Multiplex Immunofluorescence Applied to Immuno-Oncology Biomarkers in Lung Cancer

Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Abstract 5431: 3D multiplexed tissue imaging reconstruction and optimized region-of-interest (ROI) selection through deep learning model of channels embedding

A Deep Learning Approach for Histology-Based Nucleus Segmentation and Tumor Microenvironment Characterization

Nonlinear Imaging Histopathology: A Pipeline to Correlate Gold-Standard Hematoxylin and Eosin Staining With Modern Nonlinear Microscopy

Three-dimensional reconstruction of laryngeal cancer with whole organ serial immunohistochemical sections

Deep learning-based virtual staining, segmentation, and classification in label-free photoacoustic histology of human specimens