DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer

Yoni Schirris,Efstratios Gavves,Iris Nederlof,Hugo Mark Horlings,Jonas Teuwen

DOI: https://doi.org/10.1016/j.media.2022.102464

2023-06-28

Abstract:We propose a Deep learning-based weak label learning method for analyzing whole slide images (WSIs) of Hematoxylin and Eosin (H&E) stained tumor tissue not requiring pixel-level or tile-level annotations using Self-supervised pre-training and heterogeneity-aware deep Multiple Instance LEarning (DeepSMILE). We apply DeepSMILE to the task of Homologous recombination deficiency (HRD) and microsatellite instability (MSI) prediction. We utilize contrastive self-supervised learning to pre-train a feature extractor on histopathology tiles of cancer tissue. Additionally, we use variability-aware deep multiple instance learning to learn the tile feature aggregation function while modeling tumor heterogeneity. For MSI prediction in a tumor-annotated and color normalized subset of TCGA-CRC (n=360 patients), contrastive self-supervised learning improves the tile supervision baseline from 0.77 to 0.87 AUROC, on par with our proposed DeepSMILE method. On TCGA-BC (n=1041 patients) without any manual annotations, DeepSMILE improves HRD classification performance from 0.77 to 0.81 AUROC compared to tile supervision with either a self-supervised or ImageNet pre-trained feature extractor. Our proposed methods reach the baseline performance using only 40% of the labeled data on both datasets. These improvements suggest we can use standard self-supervised learning techniques combined with multiple instance learning in the histopathology domain to improve genomic label classification performance with fewer labeled data.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main focus of this paper is to predict Homologous Recombination Deficiency (HRD) and MicroSatellite Instability (MSI) in pathological images using weakly supervised learning methods to assist personalized medical decision-making. Specifically, the study attempts to address the following key issues: 1. **Limitations of existing technologies**: The primary techniques currently used to determine HRD or MSI include whole-genome sequencing, targeted DNA sequencing, and immunohistochemistry. These methods are costly, time-consuming, and not widely available globally. Additionally, the relationship between these molecular characteristics and functional defects in the homologous recombination pathway remains unclear. 2. **Diagnosis using conventional stained tissue sections (H&E staining)**: Although H&E stained tissue sections are very common in clinical diagnosis, they are not currently used to detect DDR defects due to a lack of understanding of the cellular and tissue morphological features related to DNA damage repair (DDR) defects. Therefore, the study aims to explore how to effectively use H&E stained tissue sections to identify these defects. 3. **Development of new machine learning methods**: To overcome the above limitations, the study proposes a new method called DeepSMILE, which combines self-supervised pre-training and Multiple Instance Learning (MIL) to improve the accuracy of HRD and MSI classification. This method can reduce the need for annotated data and can work without pixel-level or tile-level annotations. 4. **Improving the performance of existing methods**: By comparing with existing tile-supervised WSI-label methods, the study demonstrates that DeepSMILE can significantly enhance performance in predicting MSI and improve HRD prediction on datasets without tumor annotations. 5. **Contribution of the model**: The study introduces a feature extractor pre-trained using SimCLR for self-supervision and an extended MIL model (VarMIL) that can account for tumor heterogeneity, further improving prediction performance. In summary, the core objective of this paper is to develop a new weakly supervised learning framework to effectively predict HRD and MSI status from H&E stained tissue sections, thereby supporting the formulation of personalized medical strategies.

DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer

DeepSMILE: Self-supervised heterogeneity-aware multiple instance learning for DNA damage response defect classification directly from H&E whole-slide images

Self supervised learning improves dMMR/MSI detection from histology slides across multiple cancers

Using Whole Slide Image Representations from Self-Supervised Contrastive Learning for Melanoma Concordance Regression

Deep learning-based classification of breast cancer molecular subtypes from H&E whole-slide images

XDEEP-MSI: Explainable Bias-Rejecting Microsatellite Instability Deep Learning System In Colorectal Cancer

Evaluating the Microsatellite Instability of Colorectal Cancer Based on Multimodal Deep Learning Integrating Histopathological and Molecular Data

Contrastive learning-based histopathological features infer molecular subtypes and clinical outcomes of breast cancer from unannotated whole slide images

Deep learning and multiscale analysis for epithelial-mesenchyme segmentation and classification in breast cancer histological images

Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study

Generative and Contrastive Based Self-Supervised Learning Model for Histopathology Image Analysis.

Self-Contrastive Weakly Supervised Learning Framework for Prognostic Prediction Using Whole Slide Images

Multiresolution Self-Supervised Feature Integration via Attention Multiple Instance Learning for Histopathology Analysis

Self-supervised clustering analysis of colorectal cancer biomarkers based on multi-scale whole slides image and mass spectrometry imaging fused images

The Whole Pathological Slide Classification via Weakly Supervised Learning

Development of a Multimodal Deep Learning Model for Predicting Microsatellite Instability in Colorectal Cancer by Integrating Histopathological Images and Clinical Data

Deep learning to assess microsatellite instability directly from histopathological whole slide images in endometrial cancer

Deep contrastive learning based tissue clustering for annotation-free histopathology image analysis

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Contrastive learning-based computational histopathology predict differential expression of cancer driver genes

Deep learning identifies morphological patterns of homologous recombination deficiency in luminal breast cancers from whole slide images