Abstract:Background: Molecular stratification and tailored target therapy bring clinical benefit for patients with triple-negative breast cancer (TNBC), but it is difficult to implement comprehensive molecular testing in clinical practice. Thus, we aim to devise an approach based on digital pathology and deep learning for the molecular and prognostic stratification of TNBC.Methods: We collected digital whole slide images (WSIs) (N=425) of our previously established TNBC cohort with multi-omics data. A deep learning-based workflow was developed and applied to these WSIs to train and validate neural network models to predict multi-omics molecular features, to identify molecular subtypes, and to improve prognostic evaluation. The models that showed high prediction accuracy were further validated on the TNBC cases from TCGA (N=143).Findings: A model was first developed for automatic tissue type classification, which enabled selection of certain tissue types on WSIs for the following prediction. Numerous molecular features can be inferred from WSIs including the somatic PIK3CA mutation, germline BRCA2 mutation and PD-L1 expression. All of the four molecular subtypes of TNBC can be accurately identified based on WSIs and distinctive morphological patterns were revealed for each subtype. The addition of image features to clinical prognostic factors significantly improved the accuracy of relapse risk assessment. The models for predicting PIK3CA mutation, PD-L1 expression, TNBC subtypes and relapse risk can be well generalized to the TCGA TNBC cases. The complete prediction workflow along with validated models was modularized and deployed on an online platform, which can realize real-time one-stop prediction for newly-uploaded WSIs.Interpretation: We proposed a deep learning-based workflow and developed neural network models to predict clinically relevant information of TNBC from pathological WSIs. Our findings and established platform may enable the implementation of artificial intelligence guided precision treatment in clinical trials and future routine practice.Funding: Fudan University Shanghai Cancer Center.Declaration of Interest: None to declare Ethical Approval: The tissue samples used in our study were obtained after the approval from the FUSCC Ethics Committee.

DeePathology: Deep Multi-Task Learning for Inferring Molecular Pathology from Cancer Transcriptome

Transformer-based deep learning integrates multi-omic data with cancer pathways

Deep Biological Pathway Informed Pathology-Genomic Multimodal Survival Prediction

Exploit Spatially Resolved Transcriptomic Data to Infer Cellular Features from Pathology Imaging Data}

Integrating Multi-Modal Cancer Data Using Deep Latent Variable Path Modelling

Multi-task deep latent spaces for cancer survival and drug sensitivity prediction

Multi-task deep latent spaces for cancer survival and treatment response prediction

A systematic pan-cancer study on deep learning-based prediction of multi-omic biomarkers from routine pathology images

Pathformer: a Biological Pathway Informed Transformer for Disease Diagnosis and Prognosis Using Multi-Omics Data.

Cancer Classification With Multi-Task Deep Learning

DeepCC: a Novel Deep Learning-Based Framework for Cancer Molecular Subtype Classification

Deep Learning and Digital Pathology Power Precision Treatment of Triple-Negative Breast Cancer

Spatial Omics Driven Crossmodal Pretraining Applied to Graph-based Deep Learning for Cancer Pathology Analysis

Exploiting common patterns in diverse cancer types via multi-task learning

Deep profiling of gene expression across 18 human cancers

Multi-task Deep Convolutional Neural Network for Cancer Diagnosis

Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis

Deep learning in cancer pathology: a new generation of clinical biomarkers

Deep Learning Framework for Comprehensive Molecular and Prognostic Stratifications of Triple-Negative Breast Cancer

Abstract 6191: Simultaneous prediction of tumor microenvironment biomarkers from pathology slides using multi-task deep regression

Integrative data analysis of multi-platform cancer data with a multimodal deep learning approach