Abstract:Abstract Tissue-based sampling and diagnosis is the extraction of information from specifically limited spaces and its diagnostic significance of a certain object. Pathologists deal with issues related to tumor heterogeneity since analyzing a single sample does not necessarily capture a representative depiction of cancer, and a tissue biopsy usually only presents a small fraction of the tumor. Many multiplex tissue imaging platforms (MTIs) make the assumption that tissue microarrays (TMAs) containing small core samples of 2-dimensional (2D) tissue sections are a good approximation of bulk tumors although tumors are not 2D. However, emerging whole slide imaging (WSI) or 3D tumor atlases that employ MTIs like cyclic immunofluorescence (CyCIF) strongly challenge this assumption. In spite of the additional insight gathered by measuring the tumor microenvironment in WSI or 3D, it can be prohibitively expensive and time-consuming to process tens or hundreds of tissue sections with CyCIF. Even when resources are not limited, the criteria for region-of-interest (ROI) selection in tissues for downstream analysis remain largely qualitative and subjective as stratified sampling requires the knowledge of objects and evaluates their features. Although TMAs fail to adequately approximate whole tissue features, a theoretical subsampling of tissue exists that can best represent the tumor in the whole slide image. To address these challenges, we propose deep learning approaches to learn multi-modal image translation tasks from two aspects: 1) generative modeling approach to reconstruct 3D CyCIF representation and 2) co-embedding CyCIF image and Hematoxylin and Eosin (H&E) section to learn multi-mappings by a cross-domain translation for minimum representative ROI selection. We demonstrate that generative modeling enables a 3D virtual CyCIF reconstruction of a colorectal cancer specimen given a small subset of the imaging data at training time. By co-embedding histology and MTI features, we propose a simple convex optimization for objective ROI selection. We demonstrate the potential application of ROI selection and the efficiency of its performance with respect to cellular heterogeneity. The key takeaway is that this pipeline allows for intelligent representation from H&E images, which enables a plethora of subsequent analyses on this representation space with other multiplexed imaging platforms such as Multiplexed ion beam imaging (MIBI), Imaging Mass Cytometry (IMC) or NanoString GeoMX as only a few ROIs could be selected and analyzed using these platforms. Citation Format: Erik Burlingame, Luke Ternes, Eun Na Kim, Joe W. Gray, Young Hwan Chang. 3D multiplexed tissue imaging reconstruction and optimized region-of-interest (ROI) selection through deep learning model of channels embedding. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5431.

Toward Robust Histology-Prior Embedding for Endomicroscopy Image Classification

Context aware decision support in neurosurgical oncology based on an efficient classification of endomicroscopic data

Weakly Supervised Representation Learning for Endomicroscopy Image Analysis.

Deep Graph-Based Multimodal Feature Embedding for Endomicroscopy Image Retrieval

Multi-view Multi-modal Feature Embedding for Endomicroscopy Mosaic Classification

Transfer Recurrent Feature Learning for Endomicroscopy Image Recognition.

When Machine Vision Meets Histology: A Comparative Evaluation of Model Architecture for Classification of Histology Sections

Multi-label Recognition of Cancer-Related Lesions with Clinical Priors on White-Light Endoscopy

3D multiplexed tissue imaging reconstruction and optimized region of interest (ROI) selection through deep learning model of channels embedding

Unsupervised Feature Learning for Endomicroscopy Image Retrieval

Representative Region Based Active Learning for Histological Classification of Colorectal Cancer

Clustering-Guided Twin Contrastive Learning for Endomicroscopy Image Classification

Hybrid Deep Transfer Network and Rotational Sample Subspace Ensemble Learning for Early Cancer Detection

Classification in Histopathology: A unique deep embeddings extractor for multiple classification tasks

Abstract 5431: 3D multiplexed tissue imaging reconstruction and optimized region-of-interest (ROI) selection through deep learning model of channels embedding

A multi-resolution model for histopathology image classification and localization with multiple instance learning

A Deep Learning Approach for Tissue Spatial Quantification and Genomic Correlations of Histopathological Images

Mine Local Homogeneous Representation by Interaction Information Clustering with Unsupervised Learning in Histopathology Images

An orientation-free ring feature descriptor with stain-variability normalization for pathology image matching

Large Scale Tissue Histopathology Image Classification, Segmentation, and Visualization Via Deep Convolutional Activation Features

An attention-based multi-resolution model for prostate whole slide imageclassification and localization