Abstract:As the global incidence of cancer continues to rise rapidly, the need for swift and precise diagnoses has become increasingly pressing. Pathologists commonly rely on H&E-panCK stain pairs for various aspects of cancer diagnosis, including the detection of occult tumor cells and the evaluation of tumor budding. Nevertheless, conventional chemical staining methods suffer from notable drawbacks, such as time-intensive processes and irreversible staining outcomes. The virtual stain technique, leveraging generative adversarial network (GAN), has emerged as a promising alternative to chemical stains. This approach aims to transform biopsy scans (often H&E) into other stain types. Despite achieving notable progress in recent years, current state-of-the-art virtual staining models confront challenges that hinder their efficacy, particularly in achieving accurate staining outcomes under specific conditions. These limitations have impeded the practical integration of virtual staining into diagnostic practices. To address the goal of producing virtual panCK stains capable of replacing chemical panCK, we propose an innovative multi-model framework. Our approach involves employing a combination of Mask-RCNN (for cell segmentation) and GAN models to extract cytokeratin distribution from chemical H&E images. Additionally, we introduce a tailored dynamic GAN model to convert H&E images into virtual panCK stains, integrating the derived cytokeratin distribution. Our framework is motivated by the fact that the unique pattern of the panCK is derived from cytokeratin distribution. As a proof of concept, we employ our virtual panCK stains to evaluate tumor budding in 45 H&E whole-slide images taken from breast cancer-invaded lymph nodes . Through thorough validation by both pathologists and the QuPath software, our virtual panCK stains demonstrate a remarkable level of accuracy. In stark contrast, the accuracy of state-of-the-art single cycleGAN virtual panCK stains is negligible. To our best knowledge, this is the first instance of a multi-model virtual panCK framework and the utilization of virtual panCK for tumor budding assessment. Our framework excels in generating dependable virtual panCK stains with significantly improved efficiency, thereby considerably reducing turnaround times in diagnosis. Furthermore, its outcomes are easily comprehensible even to pathologists who may not be well-versed in computer technology. We firmly believe that our framework has the potential to advance the field of virtual stain, thereby making significant strides towards improved cancer diagnosis.

Uncertainty-assisted Virtual Immunohistochemical Detection on Morphological Staining Via Semi-Supervised Learning

TP53 Co-Mutational Features and NGS-Calibrated Immunohistochemistry Threshold in Gastric Cancer

Virtual Immunohistochemistry Staining for Histological Images Assisted by Weakly-supervised Learning

Automatic Scoring Method for Tumor IHC Images Based on Deep Learning and Its Application on P53 Protein.

Virtual multiplexed immunofluorescence staining from non-antibody-stained fluorescence imaging for gastric cancer prognosis

Transformation from hematoxylin-and-eosin staining to Ki-67 immunohistochemistry digital staining images using deep learning: experimental validation on the labeling index

A Deep Learning Quantification Algorithm for HER2 Scoring of Gastric Cancer

Histology Virtual Staining with Mask-Guided Adversarial Transfer Learning for Tertiary Lymphoid Structure Detection

Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

Histopathological bladder cancer gene mutation prediction with hierarchical deep multiple-instance learning

Evaluation of tumor budding with virtual panCK stains generated by novel multi-model CNN framework

Prediction of TP53 mutations by p53 immunohistochemistry and their prognostic significance in gastric cancer

Generalizing AI-driven Assessment of Immunohistochemistry across Immunostains and Cancer Types: A Universal Immunohistochemistry Analyzer

A Virtual Staining Method for Immunohistochemical Images of Breast Cancer.

Pathological Semantics-Preserving Learning for H&E-to-IHC Virtual Staining

Prediction of Microsatellite Instability From Gastric Histological Images Based on Residual Attention Networks With Non-Local Modules

Application of Automated Segmentation and Classification of PD-L1 Immunohistochemistry Images in Cancer Treatment Prediction.

Unpaired H&E to PR Stain Transfer with Self-Supervised Auxiliary Segmentation.

Prediction of Tumor-Associated Macrophages and Immunotherapy Benefits Using Weakly Supervised Contrastive Learning in Breast Cancer Pathology Images

Virtual Multiplexed Immunofluorescence Fl Uorescence Staining from Non-Antibody-stained Fl Uorescence Imaging for Gastric Cancer Prognosis

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks