[Clinical application of various stem cells].

T. Nakahata

2008-05-01

Abstract:

What problem does this paper attempt to address?

Stain Style Transfer of Histopathology Images Via Structure-Preserved Generative Learning

Hanwen Liang,Konstantinos N. Plataniotis,Xingyu Li

DOI: https://doi.org/10.48550/arXiv.2007.12578

2020-07-24

Image and Video Processing

Abstract:Computational histopathology image diagnosis becomes increasingly popular and important, where images are segmented or classified for disease diagnosis by computers. While pathologists do not struggle with color variations in slides, computational solutions usually suffer from this critical issue. To address the issue of color variations in histopathology images, this study proposes two stain style transfer models, SSIM-GAN and DSCSI-GAN, based on the generative adversarial networks. By cooperating structural preservation metrics and feedback of an auxiliary diagnosis net in learning, medical-relevant information presented by image texture, structure, and chroma-contrast features is preserved in color-normalized images. Particularly, the smart treat of chromatic image content in our DSCSI-GAN model helps to achieve noticeable normalization improvement in image regions where stains mix due to histological substances co-localization. Extensive experimentation on public histopathology image sets indicates that our methods outperform prior arts in terms of generating more stain-consistent images, better preserving histological information in images, and obtaining significantly higher learning efficiency. Our python implementation is published on https://github.com/hanwen0529/DSCSI-GAN.
Generative Adversarial Networks for Stain Normalisation in Histopathology

Jack Breen,Kieran Zucker,Katie Allen,Nishant Ravikumar,Nicolas M. Orsi

2024-03-07

Abstract:The rapid growth of digital pathology in recent years has provided an ideal opportunity for the development of artificial intelligence-based tools to improve the accuracy and efficiency of clinical diagnoses. One of the significant roadblocks to current research is the high level of visual variability across digital pathology images, causing models to generalise poorly to unseen data. Stain normalisation aims to standardise the visual profile of digital pathology images without changing the structural content of the images. In this chapter, we explore different techniques which have been used for stain normalisation in digital pathology, with a focus on approaches which utilise generative adversarial networks (GANs). Typically, GAN-based methods outperform non-generative approaches but at the cost of much greater computational requirements. However, it is not clear which method is best for stain normalisation in general, with different GAN and non-GAN approaches outperforming each other in different scenarios and according to different performance metrics. This is an ongoing field of study as researchers aim to identify a method which efficiently and effectively normalises pathology images to make AI models more robust and generalisable.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
Self-Supervised Digital Histopathology Image Disentanglement for Arbitrary Domain Stain Transfer.

Yu Ling,Weimin Tan,Bo Yan

DOI: https://doi.org/10.1109/tmi.2023.3298361

IF: 10.6

2023-01-01

IEEE Transactions on Medical Imaging

Abstract:Diagnosis of cancerous diseases relies on digital histopathology images from stained slides. However, the staining varies among medical centers, which leads to a domain gap of staining. Existing generative adversarial network (GAN) based stain transfer methods highly rely on distinct domains of source and target, and cannot handle unseen domains. To overcome these obstacles, we propose a self-supervised disentanglement network (SDN) for domain-independent optimization and arbitrary domain stain transfer. SDN decomposes an image into features of content and stain. By exchanging the stain features, the staining style of an image is transferred to the target domain. For optimization, we propose a novel self-supervised learning policy based on the consistency of stain and content among augmentations from one instance. Therefore, the process of training SDN is independent on the domain of training data, and thus SDN is able to tackle unseen domains. Exhaustive experiments demonstrate that SDN achieves the top performance in intra-dataset and cross-dataset stain transfer compared with the state-of-the-art stain transfer models, while the number of parameters in SDN is three orders of magnitude smaller parameters than that of compared models. Through stain transfer, SDN improves AUC of downstream classification model on unseen data without fine-tuning. Therefore, the proposed disentanglement framework and self-supervised learning policy have significant advantages in eliminating the stain gap among multi-center histopathology images.
Using machine learning for chemical-free histological tissue staining

Julie A. Renner,Patrick C. Riley

DOI: https://doi.org/10.1080/01478885.2024.2338585

2024-04-24

Journal of Histotechnology

Abstract:Hematoxylin and eosin staining can be hazardous, expensive, and prone to error and variability. To circumvent these issues, artificial intelligence/machine learning models such as generative adversarial networks (GANs), are being used to 'virtually' stain unstained tissue images indistinguishable from chemically stained tissue. Frameworks such as deep convolutional GANs (DCGAN) and conditional GANs (CGANs) have successfully generated highly reproducible 'stained' images. However, their utility may be limited by requiring registered, paired images which can be difficult to obtain. To avoid these dataset requirements, we attempted to use an unsupervised CycleGAN pix2pix model (5,6) to turn unpaired, unstained bright-field images into pathologist-approved digitally 'stained' images. Using formalin-fixed-paraffin-embedded liver samples, 5μm section images (20x) were obtained before and after staining to create "stained" an "unstained" datasets. Model implementation was conducted using Ubuntu 20.04.4 LTS, 32 GB RAM, Intel Core i7-9750 CPU @2.6 GHz, Nvidia GeForce RTX 2070 Mobile, Python 3.7.11 and Tensorflow 2.9.1. The CycleGAN framework utilized a u-net-based generator and discriminator from pix2pix, a CGAN. The CycleGAN used a modified loss function, cycle consistent loss that assumed unpaired images, so loss was measured twice. To our knowledge, this is the first documented application of this architecture using unpaired bright-field images. Results and suggested improvements are discussed.

cell biology
Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations

Sophia J. Wagner,Nadieh Khalili,Raghav Sharma,Melanie Boxberg,Carsten Marr,Walter de Back,Tingying Peng

DOI: https://doi.org/10.48550/arXiv.2107.12357

2021-07-27

Abstract:In digital pathology, different staining procedures and scanners cause substantial color variations in whole-slide images (WSIs), especially across different laboratories. These color shifts result in a poor generalization of deep learning-based methods from the training domain to external pathology data. To increase test performance, stain normalization techniques are used to reduce the variance between training and test domain. Alternatively, color augmentation can be applied during training leading to a more robust model without the extra step of color normalization at test time. We propose a novel color augmentation technique, HistAuGAN, that can simulate a wide variety of realistic histology stain colors, thus making neural networks stain-invariant when applied during training. Based on a generative adversarial network (GAN) for image-to-image translation, our model disentangles the content of the image, i.e., the morphological tissue structure, from the stain color attributes. It can be trained on multiple domains and, therefore, learns to cover different stain colors as well as other domain-specific variations introduced in the slide preparation and imaging process. We demonstrate that HistAuGAN outperforms conventional color augmentation techniques on a classification task on the publicly available dataset Camelyon17 and show that it is able to mitigate present batch effects.

Image and Video Processing,Computer Vision and Pattern Recognition
The Purification and Concentration of Diphtheria Toxin

M. D. Eaton

DOI: https://doi.org/10.1128/jb.31.4.347-366.1936

IF: 3.2

1936-04-01

Journal of Bacteriology

Abstract:
Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Shuting Liu,Baochang Zhang,Yiqing Liu,Anjia Han,Huijuan Shi,Tian Guan,Yonghong He

DOI: https://doi.org/10.1109/tmi.2021.3069874

IF: 10.6

2021-01-01

IEEE Transactions on Medical Imaging

Abstract:Pathological examination is the gold standard for the diagnosis of cancer. Common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry (IHC). In some cases, it is hard to make accurate diagnoses of cancer by referring only to H&E staining images. Whereas, the IHC examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual IHC images from H&E-stained images will be a good solution for current IHC examination hard accessibility issue, especially for some low-resource regions. However, existing approaches have limitations in microscopic structural preservation and the consistency of pathology properties. In addition, pixel-level paired data is hard available. In our work, we propose a novel adversarial learning method for effective Ki-67-stained image generation from corresponding H&E-stained image. Our method takes fully advantage of structural similarity constraint and skip connection to improve structural details preservation; and pathology consistency constraint and pathological representation network are first proposed to enforce the generated and source images hold the same pathological properties in different staining domains. We empirically demonstrate the effectiveness of our approach on two different unpaired histopathological datasets. Extensive experiments indicate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin. In addition, our approach also achieves a stable and good performance on unbalanced datasets, which shows our method has strong robustness. We believe that our method has significant potential in clinical virtual staining and advance the progress of computer-aided multi-staining histology image analysis.
Analysis of the Atrial I Wave in Patients with Chronic Obstructive Pulmonary Disease *

Juan,B.,Calatayu,José,M.,Abad,Nguyen,Bâ,Khoi,Harold,Silver

Abstract:The atrial T wave (l’a) was studied by computer analysis in 171 patients with chronic obstructive pulmonary disease (COPD). They were divided into three groups according to Ta amplitude. Group 1 included 98 cases (57.6 percent) with Ta of .05 my or less, group 2, 38 cases (22.3 percent) with Ta between .06 my and .09 my, and group 3, 35 cases (20.5 percent) with Ta of .10 my or more. The mean Ta for the total group was .05 my (range .00 my to .18 my). Distribution of the three groups according to heart rate, PR interval, P wave amplitude, duration, axis, MVV L/min., VC ml and FEy1 ml was obtained. Ta amplitude correlated well with heart rate, P wave amplitude and axis. As mean P wave axis shifted rightward, Ta amplitude increased. No correlation was found between the Ta and PR interval or P wave duration. The Ta amplitude correlated well with the pulmonary functions and patients with the greatest reduction in such functions exhibited the highest Ta values. The patients were divided into quartiles on the basis of MVV. The number of patients with Ta of .10 my or more in quartile 4 (the most severe) was almost double than in quartiles 2 and 3, and about seven times that of quartile 1. Conclusion is that Ta amplitude of .10 my or more correlated well with COPD and .06 my or more is highly suggestive.
Normalization of HE-stained histological images using cycle consistent generative adversarial networks

Marlen Runz,Daniel Rusche,Stefan Schmidt,Martin R. Weihrauch,Jürgen Hesser,Cleo-Aron Weis

DOI: https://doi.org/10.1186/s13000-021-01126-y

2021-08-06

Diagnostic Pathology

Abstract:Abstract Background Histological images show strong variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. This can impede downstream image analysis such as staining intensity evaluation or classification. Methods to reduce these variances are called image normalization techniques. Methods In this paper, we investigate the potential of CycleGAN (cycle consistent Generative Adversarial Network) for color normalization in hematoxylin-eosin stained histological images using daily clinical data with consideration of the variability of internal staining protocol variations. The network consists of a generator network G B that learns to map an image X from a source domain A to a target domain B , i.e. G B : X A → X B . In addition, a discriminator network D B is trained to distinguish whether an image from domain B is real or generated. The same process is applied to another generator-discriminator pair ( G A , D A ), for the inverse mapping G A : X B → X A . Cycle consistency ensures that a generated image is close to its original when being mapped backwards ( G A ( G B ( X A ))≈ X A and vice versa). We validate the CycleGAN approach on a breast cancer challenge and a follicular thyroid carcinoma data set for various stain variations. We evaluate the quality of the generated images compared to the original images using similarity measures. In addition, we apply stain normalization on pathological lymph node data from our institute and test the gain from normalization on a ResNet classifier pre-trained on the Camelyon16 data set. Results Qualitative results of the images generated by our network are compared to original color distributions. Our evaluation indicates that by mapping images to a target domain, the similarity training images from that domain improves up to 96%. We also achieve a high cycle consistency for the generator networks by obtaining similarity indices greater than 0.9. When applying the CycleGAN normalization to HE-stain images from our institute the kappa-value of the ResNet-model that is only trained on Camelyon16 data is increased more than 50%. Conclusions CycleGANs have proven to efficiently normalize HE-stained images. The approach compensates for deviations resulting from image acquisition (e.g. different scanning devices) as well as from tissue staining (e.g. different staining protocols), and thus overcomes the staining variations in images from various institutions.The code is publicly available at https://github.com/m4ln/stainTransfer_CycleGAN_pytorch . The data set supporting the solutions is available at 10.11588/data/8LKEZF .

pathology
F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation

Man M. Ho,Shikha Dubey,Yosep Chong,Beatrice Knudsen,Tolga Tasdizen

2024-04-19

Abstract:The Frozen Section (FS) technique is a rapid and efficient method, taking only 15-30 minutes to prepare slides for pathologists' evaluation during surgery, enabling immediate decisions on further surgical interventions. However, FS process often introduces artifacts and distortions like folds and ice-crystal effects. In contrast, these artifacts and distortions are absent in the higher-quality formalin-fixed paraffin-embedded (FFPE) slides, which require 2-3 days to prepare. While Generative Adversarial Network (GAN)-based methods have been used to translate FS to FFPE images (F2F), they may leave morphological inaccuracies with remaining FS artifacts or introduce new artifacts, reducing the quality of these translations for clinical assessments. In this study, we benchmark recent generative models, focusing on GANs and Latent Diffusion Models (LDMs), to overcome these limitations. We introduce a novel approach that combines LDMs with Histopathology Pre-Trained Embeddings to enhance restoration of FS images. Our framework leverages LDMs conditioned by both text and pre-trained embeddings to learn meaningful features of FS and FFPE histopathology images. Through diffusion and denoising techniques, our approach not only preserves essential diagnostic attributes like color staining and tissue morphology but also proposes an embedding translation mechanism to better predict the targeted FFPE representation of input FS images. As a result, this work achieves a significant improvement in classification performance, with the Area Under the Curve rising from 81.99% to 94.64%, accompanied by an advantageous CaseFD. This work establishes a new benchmark for FS to FFPE image translation quality, promising enhanced reliability and accuracy in histopathology FS image analysis. Our work is available at

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

Zhaoyang Xu,Xingru Huang,Carlos Fernández Moro,Béla Bozóky,Qianni Zhang

DOI: https://doi.org/10.48550/arXiv.1901.04059

2022-07-09

Abstract:Histopathological cancer diagnosis is based on visual examination of stained tissue slides. Hematoxylin and eosin (H\&E) is a standard stain routinely employed worldwide. It is easy to acquire and cost effective, but cells and tissue components show low-contrast with varying tones of dark blue and pink, which makes difficult visual assessments, digital image analysis, and quantifications. These limitations can be overcome by IHC staining of target proteins of the tissue slide. IHC provides a selective, high-contrast imaging of cells and tissue components, but their use is largely limited by a significantly more complex laboratory processing and high cost. We proposed a conditional CycleGAN (cCGAN) network to transform the H\&E stained images into IHC stained images, facilitating virtual IHC staining on the same slide. This data-driven method requires only a limited amount of labelled data but will generate pixel level segmentation results. The proposed cCGAN model improves the original network \cite{zhu_unpaired_2017} by adding category conditions and introducing two structural loss functions, which realize a multi-subdomain translation and improve the translation accuracy as well. % need to give reasons here. Experiments demonstrate that the proposed model outperforms the original method in unpaired image translation with multi-subdomains. We also explore the potential of unpaired images to image translation method applied on other histology images related tasks with different staining techniques.

Computer Vision and Pattern Recognition
Histopathological Stain Transfer using Style Transfer Network with Adversarial Loss

Harshal Nishar,Nikhil Chavanke,Nitin Singhal

DOI: https://doi.org/10.48550/arXiv.2010.02659

2020-10-06

Abstract:Deep learning models that are trained on histopathological images obtained from a single lab and/or scanner give poor inference performance on images obtained from another scanner/lab with a different staining protocol. In recent years, there has been a good amount of research done for image stain normalization to address this issue. In this work, we present a novel approach for the stain normalization problem using fast neural style transfer coupled with adversarial loss. We also propose a novel stain transfer generator network based on High-Resolution Network (HRNet) which requires less training time and gives good generalization with few paired training images of reference stain and test stain. This approach has been tested on Whole Slide Images (WSIs) obtained from 8 different labs, where images from one lab were treated as a reference stain. A deep learning model was trained on this stain and the rest of the images were transferred to it using the corresponding stain transfer generator network. Experimentation suggests that this approach is able to successfully perform stain normalization with good visual quality and provides better inference performance compared to not applying stain normalization.

Image and Video Processing,Computer Vision and Pattern Recognition
Enhanced Pathology Image Quality with Restore–Generative Adversarial Network

Ruichen Rong,Shidan Wang,Xinyi Zhang,Zhuoyu Wen,Xian Cheng,Liwei Jia,Yang Xie,Xiaowei Zhan,Guanghua Xiao,Donghan M. Yang

DOI: https://doi.org/10.1016/j.ajpath.2022.12.011

2023-04-01

Abstract:Whole slide imaging is becoming a routine procedure in clinical diagnosis. Advanced image analysis techniques have been developed to assist pathologists in disease diagnosis, staging, subtype classification, and risk stratification. Recently, deep learning algorithms have achieved state-of-the-art performances in various imaging analysis tasks, including tumor region segmentation, nuclei detection, and disease classification. However, widespread clinical use of these algorithms is hampered by their performances often degrading due to image quality issues commonly seen in real-world pathology imaging data such as low resolution, blurring regions, and staining variation. Restore-Generative Adversarial Network (GAN), a deep learning model, was developed to improve the imaging qualities by restoring blurred regions, enhancing low resolution, and normalizing staining colors. The results demonstrate that Restore-GAN can significantly improve image quality, which leads to improved model robustness and performance for existing deep learning algorithms in pathology image analysis. Restore-GAN has the potential to be used to facilitate the applications of deep learning models in digital pathology analyses.

pathology
Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

Christopher D. Walsh,Joanne Edwards,Robert H. Insall

DOI: https://doi.org/10.48550/arXiv.2204.06849

2022-04-14

Abstract:Immunohistochemistry is a valuable diagnostic tool for cancer pathology. However, it requires specialist labs and equipment, is time-intensive, and is difficult to reproduce. Consequently, a long term aim is to provide a digital method of recreating physical immunohistochemical stains. Generative Adversarial Networks have become exceedingly advanced at mapping one image type to another and have shown promise at inferring immunostains from haematoxylin and eosin. However, they have a substantial weakness when used with pathology images as they can fabricate structures that are not present in the original data. CycleGANs can mitigate invented tissue structures in pathology image mapping but have a related disposition to generate areas of inaccurate staining. In this paper, we describe a modification to the loss function of a CycleGAN to improve its mapping ability for pathology images by enforcing realistic stain replication while retaining tissue structure. Our approach improves upon others by considering structure and staining during model training. We evaluated our network using the Fréchet Inception distance, coupled with a new technique that we propose to appraise the accuracy of virtual immunohistochemistry. This assesses the overlap between each stain component in the inferred and ground truth images through colour deconvolution, thresholding and the Sorensen-Dice coefficient. Our modified loss function resulted in a Dice coefficient for the virtual stain of 0.78 compared with the real AE1/AE3 slide. This was superior to the unaltered CycleGAN's score of 0.74. Additionally, our loss function improved the Fréchet Inception distance for the reconstruction to 74.54 from 76.47. We, therefore, describe an advance in virtual restaining that can extend to other immunostains and tumour types and deliver reproducible, fast and readily accessible immunohistochemistry worldwide.

Image and Video Processing,Computer Vision and Pattern Recognition,Quantitative Methods
RandStainNA: Learning Stain-Agnostic Features from Histology Slides by Bridging Stain Augmentation and Normalization

Yiqing Shen,Yulin Luo,Dinggang Shen,Jing Ke

DOI: https://doi.org/10.1007/978-3-031-16434-7_21

2022-06-26

Abstract:Stain variations often decrease the generalization ability of deep learning based approaches in digital histopathology analysis. Two separate proposals, namely stain normalization (SN) and stain augmentation (SA), have been spotlighted to reduce the generalization error, where the former alleviates the stain shift across different medical centers using template image and the latter enriches the accessible stain styles by the simulation of more stain variations. However, their applications are bounded by the selection of template images and the construction of unrealistic styles. To address the problems, we unify SN and SA with a novel RandStainNA scheme, which constrains variable stain styles in a practicable range to train a stain agnostic deep learning model. The RandStainNA is applicable to stain normalization in a collection of color spaces i.e. HED, HSV, LAB. Additionally, we propose a random color space selection scheme to gain extra performance improvement. We evaluate our method by two diagnostic tasks i.e. tissue subtype classification and nuclei segmentation, with various network backbones. The performance superiority over both SA and SN yields that the proposed RandStainNA can consistently improve the generalization ability, that our models can cope with more incoming clinical datasets with unpredicted stain styles. The codes is available at <a class="link-external link-https" href="https://github.com/yiqings/RandStainNA" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification

Yuan Xue,Jiarong Ye,Qianying Zhou,Rodney Long,Sameer Antani,Zhiyun Xue,Carl Cornwell,Richard Zaino,Keith Cheng,Xiaolei Huang

DOI: https://doi.org/10.1016/j.media.2020.101816

2021-11-11

Abstract:Histopathological analysis is the present gold standard for precancerous lesion diagnosis. The goal of automated histopathological classification from digital images requires supervised training, which requires a large number of expert annotations that can be expensive and time-consuming to collect. Meanwhile, accurate classification of image patches cropped from whole-slide images is essential for standard sliding window based histopathology slide classification methods. To mitigate these issues, we propose a carefully designed conditional GAN model, namely HistoGAN, for synthesizing realistic histopathology image patches conditioned on class labels. We also investigate a novel synthetic augmentation framework that selectively adds new synthetic image patches generated by our proposed HistoGAN, rather than expanding directly the training set with synthetic images. By selecting synthetic images based on the confidence of their assigned labels and their feature similarity to real labeled images, our framework provides quality assurance to synthetic augmentation. Our models are evaluated on two datasets: a cervical histopathology image dataset with limited annotations, and another dataset of lymph node histopathology images with metastatic cancer. Here, we show that leveraging HistoGAN generated images with selective augmentation results in significant and consistent improvements of classification performance (6.7% and 2.8% higher accuracy, respectively) for cervical histopathology and metastatic cancer datasets.

Image and Video Processing,Computer Vision and Pattern Recognition
A Laplacian Pyramid Based Generative H&E Stain Augmentation Network

Fangda Li,Zhiqiang Hu,Wen Chen,Avinash Kak

2023-07-15

Abstract:Hematoxylin and Eosin (H&E) staining is a widely used sample preparation procedure for enhancing the saturation of tissue sections and the contrast between nuclei and cytoplasm in histology images for medical diagnostics. However, various factors, such as the differences in the reagents used, result in high variability in the colors of the stains actually recorded. This variability poses a challenge in achieving generalization for machine-learning based computer-aided diagnostic tools. To desensitize the learned models to stain variations, we propose the Generative Stain Augmentation Network (G-SAN) -- a GAN-based framework that augments a collection of cell images with simulated yet realistic stain variations. At its core, G-SAN uses a novel and highly computationally efficient Laplacian Pyramid (LP) based generator architecture, that is capable of disentangling stain from cell morphology. Through the task of patch classification and nucleus segmentation, we show that using G-SAN-augmented training data provides on average 15.7% improvement in F1 score and 7.3% improvement in panoptic quality, respectively. Our code is available at <a class="link-external link-https" href="https://github.com/lifangda01/GSAN-Demo" rel="external noopener nofollow">this https URL</a>.

Image and Video Processing,Computer Vision and Pattern Recognition
Impact of Stain Variation and Color Normalization for Prognostic Predictions in Pathology

Siyu,Haowen Zhou,Richard J. Cote,Mark Watson,Ramaswamy Govindan,Changhuei Yang

2024-09-13

Abstract:In recent years, deep neural networks (DNNs) have demonstrated remarkable performance in pathology applications, potentially even outperforming expert pathologists due to their ability to learn subtle features from large datasets. One complication in preparing digital pathology datasets for DNN tasks is variation in tinctorial qualities. A common way to address this is to perform stain normalization on the images. In this study, we show that a well-trained DNN model trained on one batch of histological slides failed to generalize to another batch prepared at a different time from the same tissue blocks, even when stain normalization methods were applied. This study used sample data from a previously reported DNN that was able to identify patients with early stage non-small cell lung cancer (NSCLC) whose tumors did and did not metastasize, with high accuracy, based on training and then testing of digital images from H&E stained primary tumor tissue sections processed at the same time. In this study we obtained a new series of histologic slides from the adjacent recuts of same tissue blocks processed in the same lab but at a different time. We found that the DNN trained on the either batch of slides/images was unable to generalize and failed to predict progression in the other batch of slides/images (AUC_cross-batch = 0.52 - 0.53 compared to AUC_same-batch = 0.74 - 0.81). The failure to generalize did not improve even when the tinctorial difference correction were made through either traditional color-tuning or stain normalization with the help of a Cycle Generative Adversarial Network (CycleGAN) process. This highlights the need to develop an entirely new way to process and collect consistent microscopy images from histologic slides that can be used to both train and allow for the general application of predictive DNN algorithms.

Image and Video Processing,Quantitative Methods
Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images

Mehdi Ounissi,Ilias Sarbout,Jean-Pierre Hugot,Christine Martinez-Vinson,Dominique Berrebi,Daniel Racoceanu

2024-06-27

Abstract:Chemical staining methods are dependable but require extensive time, expensive chemicals, and raise environmental concerns. These challenges highlight the need for alternative solutions like virtual staining, which accelerates the diagnostic process and enhances stain application flexibility. Generative AI technologies are pivotal in addressing these issues. However, the high-stakes nature of healthcare decisions, especially in computational pathology, complicates the adoption of these tools due to their opaque processes. Our work introduces the use of generative AI for virtual staining, aiming to enhance performance, trustworthiness, scalability, and adaptability in computational pathology. The methodology centers on a singular H&E encoder supporting multiple stain decoders. This design focuses on critical regions in the latent space of H&E, enabling precise synthetic stain generation. Our method, tested to generate 8 different stains from a single H&E slide, offers scalability by loading only necessary model components during production. We integrate label-free knowledge in training, using loss functions and regularization to minimize artifacts, thus improving paired/unpaired virtual staining accuracy. To build trust, we use real-time self-inspection with discriminators for each stain type, providing pathologists with confidence heat-maps. Automatic quality checks on new H&E slides ensure conformity to the trained distribution, ensuring accurate synthetic stains. Recognizing pathologists' challenges with new technologies, we have developed an open-source, cloud-based system, that allows easy virtual staining of H&E slides through a browser, addressing hardware/software issues and facilitating real-time user feedback. We also curated a novel dataset of 8 paired H&E/stains related to pediatric Crohn's disease, comprising 480 WSIs to further stimulate computational pathology research.

Image and Video Processing,Computer Vision and Pattern Recognition
Unified Framework for Histopathology Image Augmentation and Classification via Generative Models

Meng Li,Chaoyi Li,Can Peng,Brian C. Lovell

2024-10-13

Abstract:Deep learning techniques have become widely utilized in histopathology image classification due to their superior performance. However, this success heavily relies on the availability of substantial labeled data, which necessitates extensive and costly manual annotation by domain experts. To address this challenge, researchers have recently employed generative models to synthesize data for augmentation, thereby enhancing classification model performance. Traditionally, this involves generating synthetic data first and then training the classification model with both synthetic and real data, which creates a two-stage, time-consuming workflow. To overcome this limitation, we propose an innovative unified framework that integrates the data generation and model training stages into a unified process. Our approach utilizes a pure Vision Transformer (ViT)-based conditional Generative Adversarial Network (cGAN) model to simultaneously handle both image synthesis and classification. An additional classification head is incorporated into the cGAN model to enable simultaneous classification of histopathology images. To improve training stability and enhance the quality of generated data, we introduce a conditional class projection technique that helps maintain class separation during the generation process. We also employ a dynamic multi-loss weighting mechanism to effectively balance the losses of the classification tasks. Furthermore, our selective augmentation mechanism actively selects the most suitable generated images for data augmentation to further improve performance. Extensive experiments on histopathology datasets show that our unified synthetic augmentation framework consistently enhances the performance of histopathology image classification models.

Image and Video Processing,Computer Vision and Pattern Recognition

[Clinical application of various stem cells].

Stain Style Transfer of Histopathology Images Via Structure-Preserved Generative Learning

Generative Adversarial Networks for Stain Normalisation in Histopathology

Self-Supervised Digital Histopathology Image Disentanglement for Arbitrary Domain Stain Transfer.

Using machine learning for chemical-free histological tissue staining

Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations

The Purification and Concentration of Diphtheria Toxin

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Analysis of the Atrial I Wave in Patients with Chronic Obstructive Pulmonary Disease *

Normalization of HE-stained histological images using cycle consistent generative adversarial networks

F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation

GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

Histopathological Stain Transfer using Style Transfer Network with Adversarial Loss

Enhanced Pathology Image Quality with Restore–Generative Adversarial Network

Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

RandStainNA: Learning Stain-Agnostic Features from Histology Slides by Bridging Stain Augmentation and Normalization

Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification

A Laplacian Pyramid Based Generative H&E Stain Augmentation Network

Impact of Stain Variation and Color Normalization for Prognostic Predictions in Pathology

Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images

Unified Framework for Histopathology Image Augmentation and Classification via Generative Models