Abstract:e13593 Background: Precision oncology increasingly relies on accurate classification of the tumor microenvironment (TME), which influences therapeutic responses and prognostication in cancer. While histological observation via H&E staining remains the gold standard due to the detailed spatial context and morphological information provided, its lack of reproducibility limits its use in clinics. Next-generation sequencing (NGS) reliably provides valuable insights into the molecular landscape of tumors. Here we debut an unsupervised machine-learning approach that leverages both the rich detail in H&E slides and the reproducibility of NGS to enhance TME classification. Methods: An ML-based image analysis algorithm was developed using a convolutional neural network (CNN) deep learning model. Using over 600 images from the TCGA COAD and READ datasets and from 139 colorectal cancer (CRC) samples in an internal cohort, we trained the CNN model to curate features from small patches of whole slide images (WSIs). Then, clusters were created from these H&E WSIs without prior labeling. The clusters were validated by experienced pathologists to confirm cluster homogeneity and filter artifacts. We trained a classifier to categorize molecular features from the WSIs into TME subtypes. Performance was measured using the area under the receiver operating characteristic curve (AUC-ROC), based on the model's ability to correctly classify the Fibrotic (F) and Immune Desert (D) TME subtypes (1), as well as MSI status identified by whole exome sequencing. The F subtype corresponds with high stromal content, low immune infiltration, and the worst prognosis. The D subtype corresponds with a cold immune TME and an intermediate prognosis. Progression-free survival (PFS) data from TCGA were used to examine prognostic significance. Results: We obtained 0.86, 0.79, 0.72, and 0.71 AUC scores for predicting the ID and F TMEs, MSI status, and KRAS mutation status, respectively. PFS data analysis showed our H&E-based approach to yield similar or better prognostic significance compared to NGS-based TME classification (ID vs others, p=0.05 (H&E) and 0.2 (NGS); F vs others, p=0.07 (H&E) and 0.09 (NGS)). Our findings showed a notable correlation with RNA- and DNA-based predictions of ID and F TMEs and MSI, with enhanced precision and more robust subtype identification while achieving high embedding consistency. Conclusions: By harnessing the inherently rich detail in histological images, our H&E-based approach complements NGS-based TME analysis. Our findings advocate for the integration of advanced image analysis into the standard TME characterization workflow to refine prognostic and therapeutic strategies for CRC. We will expand this algorithm for use with other cancer types to realize its potential as a promising adjunct to NGS for precise TME classification. 1. Bagaev et al., 2021.

A network-based deep learning methodology for stratification of tumor mutations

Network Based Stratification of Major Cancers by Integrating Somatic Mutation and Gene Expression Data.

Structural deep clustering network for stratification of breast cancer patients through integration of somatic mutation profiles

Deep Neural Network Integrated into Network-Based Stratification (D3NS): A Method to Uncover Cancer Subtypes from Somatic Mutations

A Disease Network-Based Deep Learning Approach for Characterizing Melanoma.

DeepGene: an Advanced Cancer Type Classifier Based on Deep Learning and Somatic Point Mutations

Deep Learning Framework for Comprehensive Molecular and Prognostic Stratifications of Triple-Negative Breast Cancer

Integrating genetic and gene expression data in network-based stratification analysis of cancers

Network-based Distance Metric with Application to Discover Disease Subtypes in Cancer

Deep Learning for Cancer Type Classification and Driver Gene Identification

Integrated Somatic Mutation Network Diffusion Model for Stratification of Breast Cancer into Different Metabolic Mutation Subtypes

Automated exploitation of deep learning for cancer patient stratification across multiple types

DeepTpNet: A Deep Learning Approach to Identify Cancer Subtypes Based on Multi-omics Data.

Image-Based Subtype Classification for Glioblastoma Using Deep Learning: Prognostic Significance and Biologic Relevance.

Cancer classification and pathway discovery using non-negative matrix factorization

Integrating somatic mutation profiles with structural deep clustering network for metabolic stratification in pancreatic cancer: a comprehensive analysis of prognostic and genomic landscapes

Advancing Breast Cancer Subtype Prediction and Mutation Analysis: Integrating Deep Learning and Machine Learning Techniques in Genomic Research

NetNorM: Capturing cancer-relevant information in somatic exome mutation data with gene networks for cancer stratification and prognosis

An unsupervised H&E-based machine-learning approach for precise prediction of tumor microenvironment subtypes.

A deep learning approach based on multi-omics data integration to construct a risk stratification prediction model for skin cutaneous melanoma

Molecular Subtyping of Cancer Based on Distinguishing Co-Expression Modules and Machine Learning