Abstract:Benefiting from the large-scale archiving of digitized whole-slide images (WSIs), computer-aided diagnosis has been well developed to assist pathologists in decision-making. Content-based WSI retrieval can be a new approach to find highly correlated WSIs in a historically diagnosed WSI archive, which has the potential usages for assisted clinical diagnosis, medical research, and trainee education. During WSI retrieval, it is particularly challenging to encode the semantic content of histopathological images and to measure the similarity between images for interpretable results due to the gigapixel size of WSIs. In this work, we propose a Retrieval with Clustering-guided Contrastive Learning (RetCCL) framework for robust and accurate WSI-level image retrieval, which integrates a novel self-supervised feature learning method and a global ranking and aggregation algorithm for much improved performance. The proposed feature learning method makes use of existing large-scale unlabeled histopathological image data, which helps learn universal features that could be used directly for subsequent WSI retrieval tasks without extra fine-tuning. The proposed WSI retrieval method not only returns a set of WSIs similar to a query WSI, but also highlights patches or sub-regions of each WSI that share high similarity with patches of the query WSI, which helps pathologists interpret the searching results. Our WSI retrieval framework has been evaluated on the tasks of anatomical site retrieval and cancer subtype retrieval using over 22,000 slides, and the performance exceeds other state-of-the-art methods significantly (around 10% for the anatomic site retrieval in terms of average mMV@10). Besides, the patch retrieval using our learned feature representation offers a performance improvement of 24% on the TissueNet dataset in terms of mMV@5 compared with using ImageNet pre-trained features, which further demonstrates the effectiveness of the proposed CCL feature learning method.

Content-based Histopathological Image Retrieval for Whole Slide Image Database Using Binary Codes

A Retrieval System For 3d Multi-Phase Contrast-Enhanced Ct Images Of Focal Liver Lesions Based On Combined Bags Of Visual Words And Texture Words

Multistage and Multi-features Medical Image Retrieval System

Combined Density, Texture and Shape Features of Multi-phase Contrast-Enhanced CT Images for CBIR of Focal Liver Lesions: A Preliminary Study

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval

Encoding histopathology whole slide images with location-aware graphs for diagnostically relevant regions retrieval

RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval

Lifelong Histopathology Whole Slide Image Retrieval via Distance Consistency Rehearsal

Histopathological Image Deep Feature Representation for CBIR in Smart PACS

High-Order Correlation-Guided Slide-Level Histology Retrieval with Self-Supervised Hashing.

Analysis and Validation of Image Search Engines in Histopathology

Content-Based Image Retrieval In Picture Archiving And Communication System

Multi-level Magnification Correlation Hashing for Scalable Histopathological Image Retrieval

Content Based Image Retrieval (CBIR) in Remote Clinical Diagnosis and Healthcare

LEARNING BINARY SEMANTIC EMBEDDING FOR BREAST HISTOLOGY IMAGE CLASSIFICATION AND RETRIEVAL

Fast and Scalable Image Search For Histology

Content-Based Image Retrieval in PACS

Automatic Whole Slide Pathology Image Diagnosis Framework Via Unit Stochastic Selection and Attention Fusion

Building High Dimensional Imaging Database for Content Based Image Search

Design and implementation of content-based medical image retrieval system on mammograms

WWFedCBMIR: World-Wide Federated Content-Based Medical Image Retrieval