Abstract:As the global incidence of cancer continues to rise rapidly, the need for swift and precise diagnoses has become increasingly pressing. Pathologists commonly rely on H&E-panCK stain pairs for various aspects of cancer diagnosis, including the detection of occult tumor cells and the evaluation of tumor budding. Nevertheless, conventional chemical staining methods suffer from notable drawbacks, such as time-intensive processes and irreversible staining outcomes. The virtual stain technique, leveraging generative adversarial network (GAN), has emerged as a promising alternative to chemical stains. This approach aims to transform biopsy scans (often H&E) into other stain types. Despite achieving notable progress in recent years, current state-of-the-art virtual staining models confront challenges that hinder their efficacy, particularly in achieving accurate staining outcomes under specific conditions. These limitations have impeded the practical integration of virtual staining into diagnostic practices. To address the goal of producing virtual panCK stains capable of replacing chemical panCK, we propose an innovative multi-model framework. Our approach involves employing a combination of Mask-RCNN (for cell segmentation) and GAN models to extract cytokeratin distribution from chemical H&E images. Additionally, we introduce a tailored dynamic GAN model to convert H&E images into virtual panCK stains, integrating the derived cytokeratin distribution. Our framework is motivated by the fact that the unique pattern of the panCK is derived from cytokeratin distribution. As a proof of concept, we employ our virtual panCK stains to evaluate tumor budding in 45 H&E whole-slide images taken from breast cancer-invaded lymph nodes . Through thorough validation by both pathologists and the QuPath software, our virtual panCK stains demonstrate a remarkable level of accuracy. In stark contrast, the accuracy of state-of-the-art single cycleGAN virtual panCK stains is negligible. To our best knowledge, this is the first instance of a multi-model virtual panCK framework and the utilization of virtual panCK for tumor budding assessment. Our framework excels in generating dependable virtual panCK stains with significantly improved efficiency, thereby considerably reducing turnaround times in diagnosis. Furthermore, its outcomes are easily comprehensible even to pathologists who may not be well-versed in computer technology. We firmly believe that our framework has the potential to advance the field of virtual stain, thereby making significant strides towards improved cancer diagnosis.

Label-free evaluation of lung and heart transplant biopsies using virtual staining

Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks

Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue

Unpaired Virtual Histological Staining Using Prior-Guided Generative Adversarial Networks.

Virtual histological staining of unlabeled autopsy tissue

Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

Virtual stain transfer in histology via cascaded deep neural networks

Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Autofluorescence Virtual Staining System for H&E Histology and Multiplex Immunofluorescence Applied to Immuno-Oncology Biomarkers in Lung Cancer

Deep learning-enabled virtual histological staining of biological samples

Deep Learning for Virtual Histological Staining of Bright-Field Microscopic Images of Unlabeled Carotid Artery Tissue

Super-resolved virtual staining of label-free tissue using diffusion models

Prediction of heart transplant rejection from routine pathology slides with self-supervised deep learning

MVFStain: Multiple virtual functional stain histopathology images generation based on specific domain mapping

Deep learning-based virtual staining, segmentation, and classification in label-free photoacoustic histology of human specimens

Evaluation of tumor budding with virtual panCK stains generated by novel multi-model CNN framework

Improving Heart Transplant Rejection Classification Training using Progressive Generative Adversarial Networks

Lung Cancer Diagnosis on Virtual Histologically Stained Tissue Using Weakly Supervised Learning

Virtual Histology with Photon Absorption Remote Sensing using a Cycle-Consistent Generative Adversarial Network with Weakly Registered Pairs

A Machine Learning Algorithm Improves the Diagnostic Accuracy of the Histologic Component of Antibody Mediated Rejection (AMR-H) in Cardiac Transplant Endomyocardial Biopsies.

Unstained Tissue Imaging and Virtual Hematoxylin and Eosin Staining of Histologic Whole Slide Images