Abstract:Digital whole slide images (WSIs) are generally captured at microscopic resolution and encompass extensive spatial data. Directly feeding these images to deep learning models is computationally intractable due to memory constraints, while downsampling the WSIs risks incurring information loss. Alternatively, splitting the WSIs into smaller patches may result in a loss of important contextual information. In this paper, we propose a novel dual attention approach, consisting of two main components, both inspired by the visual examination process of a pathologist: The first soft attention model processes a low magnification view of the WSI to identify relevant regions of interest, followed by a custom sampling method to extract diverse and spatially distinct image tiles from the selected ROIs. The second component, the hard attention classification model further extracts a sequence of multi-resolution glimpses from each tile for classification. Since hard attention is non-differentiable, we train this component using reinforcement learning to predict the location of the glimpses. This approach allows the model to focus on essential regions instead of processing the entire tile, thereby aligning with a pathologist's way of diagnosis. The two components are trained in an end-to-end fashion using a joint loss function to demonstrate the efficacy of the model. The proposed model was evaluated on two WSI-level classification problems: Human epidermal growth factor receptor 2 scoring on breast cancer histology images and prediction of Intact/Loss status of two Mismatch Repair biomarkers from colorectal cancer histology images. We show that the proposed model achieves performance better than or comparable to the state-of-the-art methods while processing less than 10% of the WSI at the highest magnification and reducing the time required to infer the WSI-level label by more than 75%.

RCNN for Region of Interest Detection in Whole Slide Images

Selecting Regions of Interest in Large Multi-Scale Images for Cancer Pathology

Fast Regions-of-Interest Detection in Whole Slide Histopathology Images.

Finding Regions of Interest in Whole Slide Images Using Multiple Instance Learning

Robust whole slide image analysis for cervical cancer screening using deep learning

Efficient Quality Control of Whole Slide Pathology Images with Human-in-the-loop Training

Region of Interest Detection in Melanocytic Skin Tumor Whole Slide Images -- Nevus & Melanoma

Region of Interest Detection in Melanocytic Skin Tumor Whole Slide Images-Nevus and Melanoma

Dual Resolution Deep Learning Network with Self-Attention Mechanism for Classification and Localisation of Colorectal Cancer in Histopathological Images

Detection of Cervical Cancer Cells in Whole Slide Images Using Deformable and Global Context Aware Faster RCNN-FPN

Overcoming the limitations of patch-based learning to detect cancer in whole slide images

Sampling Based Tumor Recognition in Whole-Slide Histology Image with Deep Learning Approaches.

Deep Learning Classification of Colorectal Lesions Based on Whole Slide Images

Whole slide semantic segmentation: large scale active learning for digital pathology

Robust ROI Detection in Whole Slide Images Guided by Pathologists' Viewing Patterns

Data-efficient and weakly supervised computational pathology on whole-slide images

Deep Learning-Based Sparse Whole-Slide Image Analysis for the Diagnosis of Gastric Intestinal Metaplasia

Multi-class Breast Tumor Region Detection for Gigapixel Pathology Images Using Deep Neural Network with Rescale Approach

Dual Attention Model with Reinforcement Learning for Classification of Histology Whole-Slide Images

A Precision Diagnostic Framework of Renal Cell Carcinoma on Whole-Slide Images using Deep Learning.

Breast ultrasound region of interest detection and lesion localisation