Abstract:Image-to-image translation (I2I) methods allow the generation of artificial images that share the content of the original image but have a different style. With the advances in Generative Adversarial Networks (GANs)-based methods, I2I methods enabled the generation of artificial images that are indistinguishable from natural images. Recently, I2I methods were also employed in histopathology for generating artificial images of in silico stained tissues from a different type of staining. We refer to this process as stain transfer. The number of I2I variants is constantly increasing, which makes a well justified choice of the most suitable I2I methods for stain transfer challenging. In our work, we compare twelve stain transfer approaches, three of which are based on traditional and nine on GAN-based image processing methods. The analysis relies on complementary quantitative measures for the quality of image translation, the assessment of the suitability for deep learning-based tissue grading, and the visual evaluation by pathologists. Our study highlights the strengths and weaknesses of the stain transfer approaches, thereby allowing a rational choice of the underlying I2I algorithms. Code, data, and trained models for stain transfer between H&E and Masson's Trichrome staining will be made available online.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the image conversion problem between different staining methods in histopathology, namely "stain transfer". Specifically, the author compared 12 stain transfer methods, including three traditional methods and nine methods based on generative adversarial networks (GANs), in order to generate high - quality artificial images between different staining types. These images can maintain the content information of the original image while changing its style or appearance. ### Background and Motivation With the development of generative adversarial network (GANs) technology, image - to - image translation (I2I) methods have been able to generate artificial images that are indistinguishable from natural images. In the field of histopathology, this technology is used to generate virtual stained images of one staining type from images of another staining type, a process known as "stain transfer". Since there are multiple I2I methods, choosing the most suitable method for stain transfer has become challenging. ### Research Objectives 1. **Compare different stain transfer methods**: Compare the performance of 12 stain transfer methods (including three traditional methods and nine GAN - based methods) through quantitative evaluation of the quality of image translation, evaluation of the applicability of deep - learning - based tissue grading, and visual evaluation by pathologists. 2. **Provide method selection guidance**: Provide a basis for researchers and practitioners to choose the most appropriate I2I algorithm by analyzing the advantages and limitations of different methods. ### Methods and Datasets - **Methods**: The study used multiple GAN - based methods, including CycleGAN, MUNIT, StainGAN, etc., as well as traditional ColorStat, Macenko method and Vahadane method. - **Datasets**: The experiment used whole - slide images (WSIs) from mouse liver tissue, which were stained with hematoxylin - eosin (H&E) and Masson's Trichrome (MT) staining methods respectively. ### Evaluation Metrics - **Structural Similarity Index (SSIM)**: Evaluate the structural similarity between the generated image and the target image. - **First Wasserstein Distance (WD)**: Measure the color distribution difference between the generated image and the target image. - **Fréchet Inception Distance (FID)**: Evaluate the texture and color similarity between the generated image and the target image. ### Main Findings - **CycleGAN performs best**: On the validation set, CycleGAN performs excellently in imitating the structure and color of the target image, followed by CUT and MUNIT. - **Traditional methods perform poorly**: Traditional pixel - to - pixel methods (such as StainNet and traditional methods) perform poorly in the stain transfer task, especially in generating complex color patterns. - **Robustness of computer - aided grading systems**: When using the generated MT images instead of the real MT images, the deep - learning - based non - alcoholic fatty liver disease grading system can still maintain high accuracy. ### Conclusions CycleGAN provides the highest quality in the stain transfer task, and the introduced distortion is comparable to or lower than that of traditional pixel - to - pixel methods. In contrast, traditional pixel - to - pixel methods are not suitable for stain transfer. Moreover, all methods derived from CycleGAN do not show more advantages than the original CycleGAN. ### Practical Applications The results of this study provide theoretical support and technical guidance for pathologists and computer - aided evaluation systems to use stain transfer methods when a certain staining type is lacking. The research team also plans to further explore stain transfer from image patches to whole - slide images, as well as conversion between different staining types, in order to accelerate the diagnosis process of pathologists.

A comparative evaluation of image-to-image translation methods for stain transfer in histopathology

Dual contrastive learning based image-to-image translation of unstained skin tissue into virtually stained H&E images

Pix2Pix-based Stain-to-Stain Translation: A Solution for Robust Stain Normalization in Histopathology Images Analysis

Stain Style Transfer of Histopathology Images Via Structure-Preserved Generative Learning

GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

Weakly Unpaired Image Translation from Hematoxylin and Eosin Staining Image to Immunohistochemistry Staining Image.

Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

Adaptive Supervised PatchNCE Loss for Learning H&E-to-IHC Stain Translation with Inconsistent Groundtruth Image Pairs

Improving Unsupervised Stain-To-Stain Translation using Self-Supervision and Meta-Learning

StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining

Virtual stain transfer in histology via cascaded deep neural networks

StainDiff: Transfer Stain Styles of Histology Images with Denoising Diffusion Probabilistic Models and Self-ensemble

MVFStain: Multiple virtual functional stain histopathology images generation based on specific domain mapping

ReStainGAN: Leveraging IHC to IF Stain Domain Translation for in-silico Data Generation

Region-guided CycleGANs for Stain Transfer in Whole Slide Images

Using machine learning for chemical-free histological tissue staining

Towards histopathological stain invariance by Unsupervised Domain Augmentation using generative adversarial networks

Comparison of deep learning models for digital H&E staining from unpaired label-free multispectral microscopy images

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images

The role of unpaired image-to-image translation for stain color normalization in colorectal cancer histology classification