Abstract:The expanding adoption of digital pathology has enabled the curation of large repositories of histology whole slide images (WSIs), which contain a wealth of information. Similar pathology image search offers the opportunity to comb through large historical repositories of gigapixel WSIs to identify cases with similar morphological features and can be particularly useful for diagnosing rare diseases, identifying similar cases for predicting prognosis, treatment outcomes, and potential clinical trial success. A critical challenge in developing a WSI search and retrieval system is scalability, which is uniquely challenging given the need to search a growing number of slides that each can consist of billions of pixels and are several gigabytes in size. Such systems are typically slow and retrieval speed often scales with the size of the repository they search through, making their clinical adoption tedious and are not feasible for repositories that are constantly growing. Here we present Fast Image Search for Histopathology (FISH), a histology image search pipeline that is infinitely scalable and achieves constant search speed that is independent of the image database size while being interpretable and without requiring detailed annotations. FISH uses self-supervised deep learning to encode meaningful representations from WSIs and a Van Emde Boas tree for fast search, followed by an uncertainty-based ranking algorithm to retrieve similar WSIs. We evaluated FISH on multiple tasks and datasets with over 22,000 patient cases spanning 56 disease subtypes. We additionally demonstrate that FISH can be used to assist with the diagnosis of rare cancer types where sufficient cases may not be available to train traditional supervised deep models. FISH is available as an easy-to-use, open-source software package (<a class="link-external link-https" href="https://github.com/mahmoodlab/FISH" rel="external noopener nofollow">this https URL</a>).

On Image Search in Histopathology

Analysis and Validation of Image Search Engines in Histopathology

On Validation of Search & Retrieval of Tissue Images in Digital Pathology

Histopathology Slide Indexing and Search: Are We There Yet?

Similar image search for histopathology: SMILY

Studying the Effect of Digital Stain Separation of Histopathology Images on Image Search Performance

Immunohistochemistry Biomarkers-Guided Image Search for Histopathology

Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review—Current Status and Future Potential

Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence

Current Landscape of Advanced Imaging Tools for Pathology Diagnostics

Objective Diagnosis for Histopathological Images Based on Machine Learning Techniques: Classical Approaches and New Trends

Fast and Scalable Image Search For Histology

Machine learning in computational histopathology: Challenges and opportunities

Histopathological Images Analysis and Predictive Modeling Implemented in Digital Pathology—Current Affairs and Perspectives

Virtual Staining of Nonfixed Tissue Histology

Computational Pathology: Challenges and Promises for Tissue Analysis

Computer analysis of histopathological images for tumor grading

Advances in Digital Pathology: From Artificial Intelligence to Label-Free Imaging

Optimization of whole slide imaging scan settings for computer vision using human lung cancer tissue

Current status and prospects for quantitative analysis of digital image of pathological specimen using image processing software including artificial intelligence

Multimodal Pathology Image Search Between H&E Slides and Multiplexed Immunofluorescent Images