Abstract:e13593 Background: Precision oncology increasingly relies on accurate classification of the tumor microenvironment (TME), which influences therapeutic responses and prognostication in cancer. While histological observation via H&E staining remains the gold standard due to the detailed spatial context and morphological information provided, its lack of reproducibility limits its use in clinics. Next-generation sequencing (NGS) reliably provides valuable insights into the molecular landscape of tumors. Here we debut an unsupervised machine-learning approach that leverages both the rich detail in H&E slides and the reproducibility of NGS to enhance TME classification. Methods: An ML-based image analysis algorithm was developed using a convolutional neural network (CNN) deep learning model. Using over 600 images from the TCGA COAD and READ datasets and from 139 colorectal cancer (CRC) samples in an internal cohort, we trained the CNN model to curate features from small patches of whole slide images (WSIs). Then, clusters were created from these H&E WSIs without prior labeling. The clusters were validated by experienced pathologists to confirm cluster homogeneity and filter artifacts. We trained a classifier to categorize molecular features from the WSIs into TME subtypes. Performance was measured using the area under the receiver operating characteristic curve (AUC-ROC), based on the model's ability to correctly classify the Fibrotic (F) and Immune Desert (D) TME subtypes (1), as well as MSI status identified by whole exome sequencing. The F subtype corresponds with high stromal content, low immune infiltration, and the worst prognosis. The D subtype corresponds with a cold immune TME and an intermediate prognosis. Progression-free survival (PFS) data from TCGA were used to examine prognostic significance. Results: We obtained 0.86, 0.79, 0.72, and 0.71 AUC scores for predicting the ID and F TMEs, MSI status, and KRAS mutation status, respectively. PFS data analysis showed our H&E-based approach to yield similar or better prognostic significance compared to NGS-based TME classification (ID vs others, p=0.05 (H&E) and 0.2 (NGS); F vs others, p=0.07 (H&E) and 0.09 (NGS)). Our findings showed a notable correlation with RNA- and DNA-based predictions of ID and F TMEs and MSI, with enhanced precision and more robust subtype identification while achieving high embedding consistency. Conclusions: By harnessing the inherently rich detail in histological images, our H&E-based approach complements NGS-based TME analysis. Our findings advocate for the integration of advanced image analysis into the standard TME characterization workflow to refine prognostic and therapeutic strategies for CRC. We will expand this algorithm for use with other cancer types to realize its potential as a promising adjunct to NGS for precise TME classification. 1. Bagaev et al., 2021.

Ensemble transformer-based multiple instance learning to predict pathological subtypes and tumor mutational burden from histopathological whole slide images of endometrial and colorectal cancer

Deep learning-based approach to reveal tumor mutational burden status from whole slide images across multiple cancer types

Development of Prognostic Biomarkers by TMB-Guided WSI Analysis: A Two-Step Approach

Interpretable Inference and Classification of Tissue Types in Histological Colorectal Cancer Slides Based on Ensembles Adaptive Boosting Prototype Tree.

An unsupervised H&E-based machine-learning approach for precise prediction of tumor microenvironment subtypes.

Pan-cancer Computational Histopathology Reveals Tumor Mutational Burden Status Through Weakly-Supervised Deep Learning.

Prediction of Tumor Mutation Load in Colorectal Cancer Histopathological Images Based on Deep Learning

Genomics-guided Representation Learning for Pathologic Pan-cancer Tumor Microenvironment Subtype Prediction

Predicting tumor mutation burden and VHL mutation from renal cancer pathology slides with self‐supervised deep learning

Multi-instance Multi-task Learning for Joint Clinical Outcome and Genomic Profile Predictions from the Histopathological Images

Prediction of Tumor-Associated Macrophages and Immunotherapy Benefits Using Weakly Supervised Contrastive Learning in Breast Cancer Pathology Images

Development of a Multimodal Deep Learning Model for Predicting Microsatellite Instability in Colorectal Cancer by Integrating Histopathological Images and Clinical Data

MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features

A Deep Learning Model for the Prediction of Microsatellite Instability and Pathogenic POLE Mutations in Colorectal Cancer Using Histopathologic Images

Abstract 6191: Simultaneous prediction of tumor microenvironment biomarkers from pathology slides using multi-task deep regression

Deep learning to assess microsatellite instability directly from histopathological whole slide images in endometrial cancer

A stemness-based signature with inspiring indications in discriminating the prognosis, immune response, and somatic mutation of endometrial cancer patients revealed by machine learning

Exploring prognostic biomarkers in pathological images of colorectal cancer patients via deep learning

Comprehensive characterization of tumor microenvironment in colorectal cancer via molecular analysis

Immune subtyping of melanoma whole slide images using multiple instance learning

Metabolic-Pathway-Based Subtyping in Endometrial Carcinoma: An Integrated Study Based on Multi-omics Analysis and Machine Learning Algorithms