Abstract:Benefiting from the large-scale archiving of digitized whole-slide images (WSIs), computer-aided diagnosis has been well developed to assist pathologists in decision-making. Content-based WSI retrieval can be a new approach to find highly correlated WSIs in a historically diagnosed WSI archive, which has the potential usages for assisted clinical diagnosis, medical research, and trainee education. During WSI retrieval, it is particularly challenging to encode the semantic content of histopathological images and to measure the similarity between images for interpretable results due to the gigapixel size of WSIs. In this work, we propose a Retrieval with Clustering-guided Contrastive Learning (RetCCL) framework for robust and accurate WSI-level image retrieval, which integrates a novel self-supervised feature learning method and a global ranking and aggregation algorithm for much improved performance. The proposed feature learning method makes use of existing large-scale unlabeled histopathological image data, which helps learn universal features that could be used directly for subsequent WSI retrieval tasks without extra fine-tuning. The proposed WSI retrieval method not only returns a set of WSIs similar to a query WSI, but also highlights patches or sub-regions of each WSI that share high similarity with patches of the query WSI, which helps pathologists interpret the searching results. Our WSI retrieval framework has been evaluated on the tasks of anatomical site retrieval and cancer subtype retrieval using over 22,000 slides, and the performance exceeds other state-of-the-art methods significantly (around 10% for the anatomic site retrieval in terms of average mMV@10). Besides, the patch retrieval using our learned feature representation offers a performance improvement of 24% on the TissueNet dataset in terms of mMV@5 compared with using ImageNet pre-trained features, which further demonstrates the effectiveness of the proposed CCL feature learning method.

Histopathological Whole Slide Image Analysis Using Context-Based CBIR.

Content-based Histopathological Image Retrieval for Whole Slide Image Database Using Binary Codes

A Retrieval System For 3d Multi-Phase Contrast-Enhanced Ct Images Of Focal Liver Lesions Based On Combined Bags Of Visual Words And Texture Words

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval

Histopathology WSI Encoding based on GCNs for Scalable and Efficient Retrieval of Diagnostically Relevant Regions

Combined Density, Texture and Shape Features of Multi-phase Contrast-Enhanced CT Images for CBIR of Focal Liver Lesions: A Preliminary Study

Lifelong Histopathology Whole Slide Image Retrieval via Distance Consistency Rehearsal

Size-Scalable Content-Based Histopathological Image Retrieval from Database That Consists of WSIs

Encoding histopathology whole slide images with location-aware graphs for diagnostically relevant regions retrieval

Encoding Histopathological WSIs Using GNN for Scalable Diagnostically Relevant Regions Retrieval.

Informative Retrieval Framework for Histopathology Whole Slides Images Based on Deep Hashing Network.

Histopathological Image Deep Feature Representation for CBIR in Smart PACS

Breast Histopathological Image Retrieval Based on Latent Dirichlet Allocation

RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval

High-Order Correlation-Guided Slide-Level Histology Retrieval with Self-Supervised Hashing.

An efficient context-aware approach for whole slide image classification

Retrieval of pathology image for breast cancer using PLSA model based on texture and pathological features

Histopathology Cross-Modal Retrieval Based on Dual-Transformer Network

Composite Biomarker Image for Advanced Visualization in Histopathology

Deep Learning-Based Semantic Segmentation of Non-Melanocytic Skin Tumors in Whole-Slide Histopathological Images.

A Comprehensive Review of Computer-aided Whole-slide Image Analysis: from Datasets to Feature Extraction, Segmentation, Classification, and Detection Approaches