Abstract:As the global incidence of cancer continues to rise rapidly, the need for swift and precise diagnoses has become increasingly pressing. Pathologists commonly rely on H&E-panCK stain pairs for various aspects of cancer diagnosis, including the detection of occult tumor cells and the evaluation of tumor budding. Nevertheless, conventional chemical staining methods suffer from notable drawbacks, such as time-intensive processes and irreversible staining outcomes. The virtual stain technique, leveraging generative adversarial network (GAN), has emerged as a promising alternative to chemical stains. This approach aims to transform biopsy scans (often H&E) into other stain types. Despite achieving notable progress in recent years, current state-of-the-art virtual staining models confront challenges that hinder their efficacy, particularly in achieving accurate staining outcomes under specific conditions. These limitations have impeded the practical integration of virtual staining into diagnostic practices. To address the goal of producing virtual panCK stains capable of replacing chemical panCK, we propose an innovative multi-model framework. Our approach involves employing a combination of Mask-RCNN (for cell segmentation) and GAN models to extract cytokeratin distribution from chemical H&E images. Additionally, we introduce a tailored dynamic GAN model to convert H&E images into virtual panCK stains, integrating the derived cytokeratin distribution. Our framework is motivated by the fact that the unique pattern of the panCK is derived from cytokeratin distribution. As a proof of concept, we employ our virtual panCK stains to evaluate tumor budding in 45 H&E whole-slide images taken from breast cancer-invaded lymph nodes . Through thorough validation by both pathologists and the QuPath software, our virtual panCK stains demonstrate a remarkable level of accuracy. In stark contrast, the accuracy of state-of-the-art single cycleGAN virtual panCK stains is negligible. To our best knowledge, this is the first instance of a multi-model virtual panCK framework and the utilization of virtual panCK for tumor budding assessment. Our framework excels in generating dependable virtual panCK stains with significantly improved efficiency, thereby considerably reducing turnaround times in diagnosis. Furthermore, its outcomes are easily comprehensible even to pathologists who may not be well-versed in computer technology. We firmly believe that our framework has the potential to advance the field of virtual stain, thereby making significant strides towards improved cancer diagnosis.

Automatic Generation of Pathological Benchmark Dataset from Hyperspectral Images of Double Stained Tissues

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Effective Immunohistochemistry Pathology Microscopy Image Generation Using CycleGAN

PathologyGAN: Learning deep representations of cancer tissue

Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification

Synthesis of diagnostic quality cancer pathology images by generative adversarial networks

Evaluation of tumor budding with virtual panCK stains generated by novel multi-model CNN framework

Multiple-datasets and Multiple-Label Based Color Normalization in Histopathology with Cgan.

Unsupervised Histopathology Image Synthesis

SA-GAN: Stain Acclimation Generative Adversarial Network for Histopathology Image Analysis

Generative Adversarial Networks Accurately Reconstruct Pan-Cancer Histology from Pathologic, Genomic, and Radiographic Latent Features

Using machine learning for chemical-free histological tissue staining

Synthetic Genitourinary Image Synthesis via Generative Adversarial Networks: Enhancing AI Diagnostic Precision

Synthetic Histology Images for Training AI Models: A Novel Approach to Improve Prostate Cancer Diagnosis

Virtual Immunohistochemistry Staining for Histological Images Assisted by Weakly-supervised Learning

Single Generative Networks for Stain Normalization and Quality Enhancement of Histological Images in Digital Pathology

Multi-scale self-attention generative adversarial network for pathology image restoration

Structurally constrained and pathology-aware convolutional transformer generative adversarial network for virtual histology staining of human coronary optical coherence tomography images

A Federated Learning System for Histopathology Image Analysis With an Orchestral Stain-Normalization GAN

Synthetic Genitourinary Image Synthesis via Generative Adversarial Networks: Enhancing Artificial Intelligence Diagnostic Precision

A Deep Learning Method for Automatic Evaluation of Diagnostic Information from Multi-Stained Histopathological Images