Essays in exploration: new demographic history of the late 19th-century United States.

M. Haines,B. Anderson

Abstract:

What problem does this paper attempt to address?

Dual contrastive learning based image-to-image translation of unstained skin tissue into virtually stained H&E images

Muhammad Zeeshan Asaf,Babar Rao,Muhammad Usman Akram,Sajid Gul Khawaja,Samavia Khan,Thu Minh Truong,Palveen Sekhon,Irfan J. Khan,Muhammad Shahmir Abbasi

DOI: https://doi.org/10.1038/s41598-024-52833-7

IF: 4.6

2024-01-29

Scientific Reports

Abstract:Staining is a crucial step in histopathology that prepares tissue sections for microscopic examination. Hematoxylin and eosin (H&E) staining, also known as basic or routine staining, is used in 80% of histopathology slides worldwide. To enhance the histopathology workflow, recent research has focused on integrating generative artificial intelligence and deep learning models. These models have the potential to improve staining accuracy, reduce staining time, and minimize the use of hazardous chemicals, making histopathology a safer and more efficient field. In this study, we introduce a novel three-stage, dual contrastive learning-based, image-to-image generative (DCLGAN) model for virtually applying an "H&E stain" to unstained skin tissue images. The proposed model utilizes a unique learning setting comprising two pairs of generators and discriminators. By employing contrastive learning, our model maximizes the mutual information between traditional H&E-stained and virtually stained H&E patches. Our dataset consists of pairs of unstained and H&E-stained images, scanned with a brightfield microscope at 20 × magnification, providing a comprehensive set of training and testing images for evaluating the efficacy of our proposed model. Two metrics, Fréchet Inception Distance (FID) and Kernel Inception Distance (KID), were used to quantitatively evaluate virtual stained slides. Our analysis revealed that the average FID score between virtually stained and H&E-stained images (80.47) was considerably lower than that between unstained and virtually stained slides (342.01), and unstained and H&E stained (320.4) indicating a similarity virtual and H&E stains. Similarly, the mean KID score between H&E stained and virtually stained images (0.022) was significantly lower than the mean KID score between unstained and H&E stained (0.28) or unstained and virtually stained (0.31) images. In addition, a group of experienced dermatopathologists evaluated traditional and virtually stained images and demonstrated an average agreement of 78.8% and 90.2% for paired and single virtual stained image evaluations, respectively. Our study demonstrates that the proposed three-stage dual contrastive learning-based image-to-image generative model is effective in generating virtual stained images, as indicated by quantified parameters and grader evaluations. In addition, our findings suggest that GAN models have the potential to replace traditional H&E staining, which can reduce both time and environmental impact. This study highlights the promise of virtual staining as a viable alternative to traditional staining techniques in histopathology.

multidisciplinary sciences
Role of farnesoid X receptor (FXR) in the process of differentiation of bone marrow stromal cells into osteoblasts.

Hicham Id Boufker,L. Lagneaux,H. Fayyad-kazan,B. Badran,M. Najar,M. Wiedig,G. Ghanem,G. Laurent,J. Body,F. Journé

DOI: https://doi.org/10.1016/j.bone.2011.08.013

IF: 4.626

2011-12-01

Bone

Abstract:
Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images

Mehdi Ounissi,Ilias Sarbout,Jean-Pierre Hugot,Christine Martinez-Vinson,Dominique Berrebi,Daniel Racoceanu

2024-06-27

Abstract:Chemical staining methods are dependable but require extensive time, expensive chemicals, and raise environmental concerns. These challenges highlight the need for alternative solutions like virtual staining, which accelerates the diagnostic process and enhances stain application flexibility. Generative AI technologies are pivotal in addressing these issues. However, the high-stakes nature of healthcare decisions, especially in computational pathology, complicates the adoption of these tools due to their opaque processes. Our work introduces the use of generative AI for virtual staining, aiming to enhance performance, trustworthiness, scalability, and adaptability in computational pathology. The methodology centers on a singular H&E encoder supporting multiple stain decoders. This design focuses on critical regions in the latent space of H&E, enabling precise synthetic stain generation. Our method, tested to generate 8 different stains from a single H&E slide, offers scalability by loading only necessary model components during production. We integrate label-free knowledge in training, using loss functions and regularization to minimize artifacts, thus improving paired/unpaired virtual staining accuracy. To build trust, we use real-time self-inspection with discriminators for each stain type, providing pathologists with confidence heat-maps. Automatic quality checks on new H&E slides ensure conformity to the trained distribution, ensuring accurate synthetic stains. Recognizing pathologists' challenges with new technologies, we have developed an open-source, cloud-based system, that allows easy virtual staining of H&E slides through a browser, addressing hardware/software issues and facilitating real-time user feedback. We also curated a novel dataset of 8 paired H&E/stains related to pediatric Crohn's disease, comprising 480 WSIs to further stimulate computational pathology research.

Image and Video Processing,Computer Vision and Pattern Recognition
Using machine learning for chemical-free histological tissue staining

Julie A. Renner,Patrick C. Riley

DOI: https://doi.org/10.1080/01478885.2024.2338585

2024-04-24

Journal of Histotechnology

Abstract:Hematoxylin and eosin staining can be hazardous, expensive, and prone to error and variability. To circumvent these issues, artificial intelligence/machine learning models such as generative adversarial networks (GANs), are being used to 'virtually' stain unstained tissue images indistinguishable from chemically stained tissue. Frameworks such as deep convolutional GANs (DCGAN) and conditional GANs (CGANs) have successfully generated highly reproducible 'stained' images. However, their utility may be limited by requiring registered, paired images which can be difficult to obtain. To avoid these dataset requirements, we attempted to use an unsupervised CycleGAN pix2pix model (5,6) to turn unpaired, unstained bright-field images into pathologist-approved digitally 'stained' images. Using formalin-fixed-paraffin-embedded liver samples, 5μm section images (20x) were obtained before and after staining to create "stained" an "unstained" datasets. Model implementation was conducted using Ubuntu 20.04.4 LTS, 32 GB RAM, Intel Core i7-9750 CPU @2.6 GHz, Nvidia GeForce RTX 2070 Mobile, Python 3.7.11 and Tensorflow 2.9.1. The CycleGAN framework utilized a u-net-based generator and discriminator from pix2pix, a CGAN. The CycleGAN used a modified loss function, cycle consistent loss that assumed unpaired images, so loss was measured twice. To our knowledge, this is the first documented application of this architecture using unpaired bright-field images. Results and suggested improvements are discussed.

cell biology
Single color digital H&E staining with In-and-Out Net

Mengkun Chen,Yen-Tung Liu,Fadeel Sher Khan,Matthew C. Fox,Jason S. Reichenberg,Fabiana C.P.S. Lopes,Katherine R. Sebastian,Mia K. Markey,James W. Tunnell

DOI: https://doi.org/10.1016/j.compmedimag.2024.102468

2024-11-23

Abstract:Virtual staining streamlines traditional staining procedures by digitally generating stained images from unstained or differently stained images. While conventional staining methods involve time-consuming chemical processes, virtual staining offers an efficient and low infrastructure alternative. Leveraging microscopy-based techniques, such as confocal microscopy, researchers can expedite tissue analysis without the need for physical sectioning. However, interpreting grayscale or pseudo-color microscopic images remains a challenge for pathologists and surgeons accustomed to traditional histologically stained images. To fill this gap, various studies explore digitally simulating staining to mimic targeted histological stains. This paper introduces a novel network, In-and-Out Net, specifically designed for virtual staining tasks. Based on Generative Adversarial Networks (GAN), our model efficiently transforms Reflectance Confocal Microscopy (RCM) images into Hematoxylin and Eosin (H&E) stained images. We enhance nuclei contrast in RCM images using aluminum chloride preprocessing for skin tissues. Training the model with virtual H\&E labels featuring two fluorescence channels eliminates the need for image registration and provides pixel-level ground truth. Our contributions include proposing an optimal training strategy, conducting a comparative analysis demonstrating state-of-the-art performance, validating the model through an ablation study, and collecting perfectly matched input and ground truth images without registration. In-and-Out Net showcases promising results, offering a valuable tool for virtual staining tasks and advancing the field of histological image analysis.

Computer Vision and Pattern Recognition,Medical Physics
Deep learning-based transformation of the H&E stain into special stains

Kevin de Haan,Yijie Zhang,Jonathan E. Zuckerman,Tairan Liu,Anthony E. Sisk,Miguel F. P. Diaz,Kuang-Yu Jen,Alexander Nobori,Sofia Liou,Sarah Zhang,Rana Riahi,Yair Rivenson,W. Dean Wallace,Aydogan Ozcan

DOI: https://doi.org/10.1038/s41467-021-25221-2

2021-08-13

Abstract:Pathology is practiced by visual inspection of histochemically stained slides. Most commonly, the hematoxylin and eosin (H&E) stain is used in the diagnostic workflow and it is the gold standard for cancer diagnosis. However, in many cases, especially for non-neoplastic diseases, additional "special stains" are used to provide different levels of contrast and color to tissue components and allow pathologists to get a clearer diagnostic picture. In this study, we demonstrate the utility of supervised learning-based computational stain transformation from H&E to different special stains (Masson's Trichrome, periodic acid-Schiff and Jones silver stain) using tissue sections from kidney needle core biopsies. Based on evaluation by three renal pathologists, followed by adjudication by a fourth renal pathologist, we show that the generation of virtual special stains from existing H&E images improves the diagnosis in several non-neoplastic kidney diseases sampled from 58 unique subjects. A second study performed by three pathologists found that the quality of the special stains generated by the stain transformation network was statistically equivalent to those generated through standard histochemical staining. As the transformation of H&E images into special stains can be achieved within 1 min or less per patient core specimen slide, this stain-to-stain transformation framework can improve the quality of the preliminary diagnosis when additional special stains are needed, along with significant savings in time and cost, reducing the burden on healthcare system and patients.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning,Medical Physics
Virtual stain transfer in histology via cascaded deep neural networks

Xilin Yang,Bijie Bai,Yijie Zhang,Yuzhu Li,Kevin de Haan,Tairan Liu,Aydogan Ozcan

DOI: https://doi.org/10.1021/acsphotonics.2c00932

2022-07-14

Abstract:Pathological diagnosis relies on the visual inspection of histologically stained thin tissue specimens, where different types of stains are applied to bring contrast to and highlight various desired histological features. However, the destructive histochemical staining procedures are usually irreversible, making it very difficult to obtain multiple stains on the same tissue section. Here, we demonstrate a virtual stain transfer framework via a cascaded deep neural network (C-DNN) to digitally transform hematoxylin and eosin (H&E) stained tissue images into other types of histological stains. Unlike a single neural network structure which only takes one stain type as input to digitally output images of another stain type, C-DNN first uses virtual staining to transform autofluorescence microscopy images into H&E and then performs stain transfer from H&E to the domain of the other stain in a cascaded manner. This cascaded structure in the training phase allows the model to directly exploit histochemically stained image data on both H&E and the target special stain of interest. This advantage alleviates the challenge of paired data acquisition and improves the image quality and color accuracy of the virtual stain transfer from H&E to another stain. We validated the superior performance of this C-DNN approach using kidney needle core biopsy tissue sections and successfully transferred the H&E-stained tissue images into virtual PAS (periodic acid-Schiff) stain. This method provides high-quality virtual images of special stains using existing, histochemically stained slides and creates new opportunities in digital pathology by performing highly accurate stain-to-stain transformations.

Medical Physics,Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing,Quantitative Methods
High Accuracy Tumor Diagnoses and Benchmarking of Hematoxylin and Eosin Stained Prostate Core Biopsy Images Generated by Explainable Deep Neural Networks

Aman Rana,Alarice Lowe,Marie Lithgow,Katharine Horback,Tyler Janovitz,Annacarolina Da Silva,Harrison Tsai,Vignesh Shanmugam,Hyung-Jin Yoon,Pratik Shah

DOI: https://doi.org/10.1001/jamanetworkopen.2020.5111

2019-08-02

Abstract:Histopathological diagnoses of tumors in tissue biopsy after Hematoxylin and Eosin (H&E) staining is the gold standard for oncology care. H&E staining is slow and uses dyes, reagents and precious tissue samples that cannot be reused. Thousands of native nonstained RGB Whole Slide Image (RWSI) patches of prostate core tissue biopsies were registered with their H&E stained versions. Conditional Generative Adversarial Neural Networks (cGANs) that automate conversion of native nonstained RWSI to computational H&E stained images were then trained. High similarities between computational and H&E dye stained images with Structural Similarity Index (SSIM) 0.902, Pearsons Correlation Coefficient (CC) 0.962 and Peak Signal to Noise Ratio (PSNR) 22.821 dB were calculated. A second cGAN performed accurate computational destaining of H&E dye stained images back to their native nonstained form with SSIM 0.9, CC 0.963 and PSNR 25.646 dB. A single-blind study computed more than 95% pixel-by-pixel overlap between prostate tumor annotations on computationally stained images, provided by five-board certified MD pathologists, with those on H&E dye stained counterparts. We report the first visualization and explanation of neural network kernel activation maps during H&E staining and destaining of RGB images by cGANs. High similarities between kernel activation maps of computational and H&E stained images (Mean-Squared Errors <0.0005) provide additional mathematical and mechanistic validation of the staining system. Our neural network framework thus is automated, explainable and performs high precision H&E staining and destaining of low cost native RGB images, and is computer vision and physician authenticated for rapid and accurate tumor diagnoses.

Quantitative Methods,Computer Vision and Pattern Recognition,Image and Video Processing
Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

Christopher D. Walsh,Joanne Edwards,Robert H. Insall

DOI: https://doi.org/10.48550/arXiv.2204.06849

2022-04-14

Abstract:Immunohistochemistry is a valuable diagnostic tool for cancer pathology. However, it requires specialist labs and equipment, is time-intensive, and is difficult to reproduce. Consequently, a long term aim is to provide a digital method of recreating physical immunohistochemical stains. Generative Adversarial Networks have become exceedingly advanced at mapping one image type to another and have shown promise at inferring immunostains from haematoxylin and eosin. However, they have a substantial weakness when used with pathology images as they can fabricate structures that are not present in the original data. CycleGANs can mitigate invented tissue structures in pathology image mapping but have a related disposition to generate areas of inaccurate staining. In this paper, we describe a modification to the loss function of a CycleGAN to improve its mapping ability for pathology images by enforcing realistic stain replication while retaining tissue structure. Our approach improves upon others by considering structure and staining during model training. We evaluated our network using the Fréchet Inception distance, coupled with a new technique that we propose to appraise the accuracy of virtual immunohistochemistry. This assesses the overlap between each stain component in the inferred and ground truth images through colour deconvolution, thresholding and the Sorensen-Dice coefficient. Our modified loss function resulted in a Dice coefficient for the virtual stain of 0.78 compared with the real AE1/AE3 slide. This was superior to the unaltered CycleGAN's score of 0.74. Additionally, our loss function improved the Fréchet Inception distance for the reconstruction to 74.54 from 76.47. We, therefore, describe an advance in virtual restaining that can extend to other immunostains and tumour types and deliver reproducible, fast and readily accessible immunohistochemistry worldwide.

Image and Video Processing,Computer Vision and Pattern Recognition,Quantitative Methods
Computational Histological Staining and Destaining of Prostate Core Biopsy RGB Images with Generative Adversarial Neural Networks

Aman Rana,Gregory Yauney,Alarice Lowe,Pratik Shah

DOI: https://doi.org/10.1109/ICMLA.2018.00133

2019-02-21

Abstract:Histopathology tissue samples are widely available in two states: paraffin-embedded unstained and non-paraffin-embedded stained whole slide RGB images (WSRI). Hematoxylin and eosin stain (H&E) is one of the principal stains in histology but suffers from several shortcomings related to tissue preparation, staining protocols, slowness and human error. We report two novel approaches for training machine learning models for the computational H&E staining and destaining of prostate core biopsy RGB images. The staining model uses a conditional generative adversarial network that learns hierarchical non-linear mappings between whole slide RGB image (WSRI) pairs of prostate core biopsy before and after H&E staining. The trained staining model can then generate computationally H&E-stained prostate core WSRIs using previously unseen non-stained biopsy images as input. The destaining model, by learning mappings between an H&E stained WSRI and a non-stained WSRI of the same biopsy, can computationally destain previously unseen H&E-stained images. Structural and anatomical details of prostate tissue and colors, shapes, geometries, locations of nuclei, stroma, vessels, glands and other cellular components were generated by both models with structural similarity indices of 0.68 (staining) and 0.84 (destaining). The proposed staining and destaining models can engender computational H&E staining and destaining of WSRI biopsies without additional equipment and devices.

Computer Vision and Pattern Recognition,Machine Learning
Stain Style Transfer of Histopathology Images Via Structure-Preserved Generative Learning

Hanwen Liang,Konstantinos N. Plataniotis,Xingyu Li

DOI: https://doi.org/10.48550/arXiv.2007.12578

2020-07-24

Image and Video Processing

Abstract:Computational histopathology image diagnosis becomes increasingly popular and important, where images are segmented or classified for disease diagnosis by computers. While pathologists do not struggle with color variations in slides, computational solutions usually suffer from this critical issue. To address the issue of color variations in histopathology images, this study proposes two stain style transfer models, SSIM-GAN and DSCSI-GAN, based on the generative adversarial networks. By cooperating structural preservation metrics and feedback of an auxiliary diagnosis net in learning, medical-relevant information presented by image texture, structure, and chroma-contrast features is preserved in color-normalized images. Particularly, the smart treat of chromatic image content in our DSCSI-GAN model helps to achieve noticeable normalization improvement in image regions where stains mix due to histological substances co-localization. Extensive experimentation on public histopathology image sets indicates that our methods outperform prior arts in terms of generating more stain-consistent images, better preserving histological information in images, and obtaining significantly higher learning efficiency. Our python implementation is published on https://github.com/hanwen0529/DSCSI-GAN.
Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Pranita Pradhan,Tobias Meyer,Michael Vieth,Andreas Stallmach,Maximilian Waldner,Michael Schmitt,Juergen Popp,Thomas Bocklitz

DOI: https://doi.org/10.1364/boe.415962

IF: 3.4

2021-03-23

Biomedical Optics Express

Abstract:Hematoxylin and Eosin (H&E) staining is the 'gold-standard' method in histopathology. However, standard H&E staining of high-quality tissue sections requires long sample preparation times including sample embedding, which restricts its application for 'real-time' disease diagnosis. Due to this reason, a label-free alternative technique like non-linear multimodal (NLM) imaging, which is the combination of three non-linear optical modalities including coherent anti-Stokes Raman scattering, two-photon excitation fluorescence and second-harmonic generation, is proposed in this work. To correlate the information of the NLM images with H&E images, this work proposes computational staining of NLM images using deep learning models in a supervised and an unsupervised approach. In the supervised and the unsupervised approach, conditional generative adversarial networks (CGANs) and cycle conditional generative adversarial networks (cycle CGANs) are used, respectively. Both CGAN and cycle CGAN models generate pseudo H&E images, which are quantitatively analyzed based on mean squared error, structure similarity index and color shading similarity index. The mean of the three metrics calculated for the computationally generated H&E images indicate significant performance. Thus, utilizing CGAN and cycle CGAN models for computational staining is beneficial for diagnostic applications without performing a laboratory-based staining procedure. To the author's best knowledge, it is the first time that NLM images are computationally stained to H&E images using GANs in an unsupervised manner.

radiology, nuclear medicine & medical imaging,biochemical research methods,optics
A Laplacian Pyramid Based Generative H&E Stain Augmentation Network

Fangda Li,Zhiqiang Hu,Wen Chen,Avinash Kak

2023-07-15

Abstract:Hematoxylin and Eosin (H&E) staining is a widely used sample preparation procedure for enhancing the saturation of tissue sections and the contrast between nuclei and cytoplasm in histology images for medical diagnostics. However, various factors, such as the differences in the reagents used, result in high variability in the colors of the stains actually recorded. This variability poses a challenge in achieving generalization for machine-learning based computer-aided diagnostic tools. To desensitize the learned models to stain variations, we propose the Generative Stain Augmentation Network (G-SAN) -- a GAN-based framework that augments a collection of cell images with simulated yet realistic stain variations. At its core, G-SAN uses a novel and highly computationally efficient Laplacian Pyramid (LP) based generator architecture, that is capable of disentangling stain from cell morphology. Through the task of patch classification and nucleus segmentation, we show that using G-SAN-augmented training data provides on average 15.7% improvement in F1 score and 7.3% improvement in panoptic quality, respectively. Our code is available at <a class="link-external link-https" href="https://github.com/lifangda01/GSAN-Demo" rel="external noopener nofollow">this https URL</a>.

Image and Video Processing,Computer Vision and Pattern Recognition
Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Yijie Zhang,Kevin de Haan,Yair Rivenson,Jingxi Li,Apostolos Delis,Aydogan Ozcan

DOI: https://doi.org/10.1038/s41377-020-0315-y

2020-01-21

Abstract:Histological staining is a vital step used to diagnose various diseases and has been used for more than a century to provide contrast to tissue sections, rendering the tissue constituents visible for microscopic analysis by medical experts. However, this process is time-consuming, labor-intensive, expensive and destructive to the specimen. Recently, the ability to virtually-stain unlabeled tissue sections, entirely avoiding the histochemical staining step, has been demonstrated using tissue-stain specific deep neural networks. Here, we present a new deep learning-based framework which generates virtually-stained images using label-free tissue, where different stains are merged following a micro-structure map defined by the user. This approach uses a single deep neural network that receives two different sources of information at its input: (1) autofluorescence images of the label-free tissue sample, and (2) a digital staining matrix which represents the desired microscopic map of different stains to be virtually generated at the same tissue section. This digital staining matrix is also used to virtually blend existing stains, digitally synthesizing new histological stains. We trained and blindly tested this virtual-staining network using unlabeled kidney tissue sections to generate micro-structured combinations of Hematoxylin and Eosin (H&E), Jones silver stain, and Masson's Trichrome stain. Using a single network, this approach multiplexes virtual staining of label-free tissue with multiple types of stains and paves the way for synthesizing new digital histological stains that can be created on the same tissue cross-section, which is currently not feasible with standard histochemical staining methods.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning,Medical Physics,Quantitative Methods
Unpaired Virtual Histological Staining Using Prior-Guided Generative Adversarial Networks.

Renao Yan,Qiming He,Yiqing Liu,Peng Ye,Lianghui Zhu,Shanshan Shi,Jizhou Gou,Yonghong He,Tian Guan,Guangde Zhou

DOI: https://doi.org/10.1016/j.compmedimag.2023.102185

IF: 7.422

2023-01-01

Computerized Medical Imaging and Graphics

Abstract:Fibrosis is an inevitable stage in the development of chronic liver disease and has an irreplaceable role in characterizing the degree of progression of chronic liver disease. Histopathological diagnosis is the gold standard for the interpretation of fibrosis parameters. Conventional hematoxylin-eosin (H&E) staining can only reflect the gross structure of the tissue and the distribution of hepatocytes, while Masson trichrome can highlight specific types of collagen fiber structure, thus providing the necessary structural information for fibrosis scoring. However, the expensive costs of time, economy, and patient specimens as well as the non-uniform preparation and staining process make the conversion of existing H&E staining into virtual Masson trichrome staining a solution for fibrosis evaluation. Existing translation approaches fail to extract fiber features accurately enough, and the decoder of staining is unable to converge due to the inconsistent color of physical staining. In this work, we propose a prior-guided generative adversarial network, based on unpaired data for effective Masson trichrome stained image generation from the corresponding H&E stained image. Conducted on a small training set, our method takes full advantage of prior knowledge to set up better constraints on both the encoder and the decoder. Experiments indicate the superior performance of our method that surpasses the previous approaches. For various liver diseases, our results demonstrate a high correlation between the staging of real and virtual stains (ρ=0.82; 95% CI: 0.73-0.89). In addition, our finetuning strategy is able to standardize the staining color and release the memory and computational burden, which can be employed in clinical assessment.
Pix2Pix-based Stain-to-Stain Translation: A Solution for Robust Stain Normalization in Histopathology Images Analysis

Pegah Salehi,Abdolah Chalechale

DOI: https://doi.org/10.48550/arXiv.2002.00647

2020-02-03

Image and Video Processing

Abstract:The diagnosis of cancer is mainly performed by visual analysis of the pathologists, through examining the morphology of the tissue slices and the spatial arrangement of the cells. If the microscopic image of a specimen is not stained, it will look colorless and textured. Therefore, chemical staining is required to create contrast and help identify specific tissue components. During tissue preparation due to differences in chemicals, scanners, cutting thicknesses, and laboratory protocols, similar tissues are usually varied significantly in appearance. This diversity in staining, in addition to Interpretive disparity among pathologists more is one of the main challenges in designing robust and flexible systems for automated analysis. To address the staining color variations, several methods for normalizing stain have been proposed. In our proposed method, a Stain-to-Stain Translation (STST) approach is used to stain normalization for Hematoxylin and Eosin (H&E) stained histopathology images, which learns not only the specific color distribution but also the preserves corresponding histopathological pattern. We perform the process of translation based on the pix2pix framework, which uses the conditional generator adversarial networks (cGANs). Our approach showed excellent results, both mathematically and experimentally against the state of the art methods. We have made the source code publicly available.
StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining

Tushar Kataria,Beatrice Knudsen,Shireen Y. Elhabian

2024-03-18

Abstract:Hematoxylin and Eosin (H&E) staining is the most commonly used for disease diagnosis and tumor recurrence tracking. Hematoxylin excels at highlighting nuclei, whereas eosin stains the cytoplasm. However, H&E stain lacks details for differentiating different types of cells relevant to identifying the grade of the disease or response to specific treatment variations. Pathologists require special immunohistochemical (IHC) stains that highlight different cell types. These stains help in accurately identifying different regions of disease growth and their interactions with the cell's microenvironment. The advent of deep learning models has made Image-to-Image (I2I) translation a key research area, reducing the need for expensive physical staining processes. Pix2Pix and CycleGAN are still the most commonly used methods for virtual staining applications. However, both suffer from hallucinations or staining irregularities when H&E stain has less discriminate information about the underlying cells IHC needs to highlight (e.g.,CD3 lymphocytes). Diffusion models are currently the state-of-the-art models for image generation and conditional generation tasks. However, they require extensive and diverse datasets (millions of samples) to converge, which is less feasible for virtual staining applications.Inspired by the success of multitask deep learning models for limited dataset size, we propose StainDiffuser, a novel multitask dual diffusion architecture for virtual staining that converges under a limited training budget. StainDiffuser trains two diffusion processes simultaneously: (a) generation of cell-specific IHC stain from H&E and (b) H&E-based cell segmentation using coarse segmentation only during training. Our results show that StainDiffuser produces high-quality results for easier (CK8/18,epithelial marker) and difficult stains(CD3, Lymphocytes).

Image and Video Processing,Computer Vision and Pattern Recognition
Virtual multi-staining in a single-section view for renal pathology using generative adversarial networks

Masataka Kawai,Toru Odate,Kazunari Kasai,Tomohiro Inoue,Kunio Mochizuki,Naoki Oishi,Tetsuo Kondo

DOI: https://doi.org/10.1016/j.compbiomed.2024.109149

2024-09-18

Abstract:Sections stained in periodic acid-Schiff (PAS), periodic acid-methenamine silver (PAM), hematoxylin and eosin (H&E), and Masson's trichrome (MT) stain with minimal morphological discordance are helpful for pathological diagnosis in renal biopsy. Here, we propose an artificial intelligence-based re-stainer called PPHM-GAN (PAS, PAM, H&E, and MT-generative adversarial networks) with multi-stain to multi-stain transformation capability. We trained three GAN models on 512 × 512-pixel patches from 26 training cases. The model with the best transformation quality was selected for each pair of stain transformations by human evaluation. Frechet inception distances, peak signal-to-noise ratio, structural similarity index measure, contrast structural similarity, and newly introduced domain shift inception score were calculated as auxiliary quality metrics. We validated the diagnostic utility using 5120 × 5120 patches of ten validation cases for major glomerular and interstitial abnormalities. Transformed stains were sometimes superior to original stains for the recognition of crescent formation, mesangial hypercellularity, glomerular sclerosis, interstitial lesions, or arteriosclerosis. 23 of 24 glomeruli (95.83 %) from 9 additional validation cases transformed to PAM, PAS, or MT facilitated recognition of crescent formation. Stain transformations to PAM (p = 4.0E-11) and transformations from H&E (p = 4.8E-9) most improved crescent formation recognition. PPHM-GAN maximizes information from a given section by providing several stains in a virtual single-section view, and may change the staining and diagnostic strategy.
Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations

Sophia J. Wagner,Nadieh Khalili,Raghav Sharma,Melanie Boxberg,Carsten Marr,Walter de Back,Tingying Peng

DOI: https://doi.org/10.48550/arXiv.2107.12357

2021-07-27

Abstract:In digital pathology, different staining procedures and scanners cause substantial color variations in whole-slide images (WSIs), especially across different laboratories. These color shifts result in a poor generalization of deep learning-based methods from the training domain to external pathology data. To increase test performance, stain normalization techniques are used to reduce the variance between training and test domain. Alternatively, color augmentation can be applied during training leading to a more robust model without the extra step of color normalization at test time. We propose a novel color augmentation technique, HistAuGAN, that can simulate a wide variety of realistic histology stain colors, thus making neural networks stain-invariant when applied during training. Based on a generative adversarial network (GAN) for image-to-image translation, our model disentangles the content of the image, i.e., the morphological tissue structure, from the stain color attributes. It can be trained on multiple domains and, therefore, learns to cover different stain colors as well as other domain-specific variations introduced in the slide preparation and imaging process. We demonstrate that HistAuGAN outperforms conventional color augmentation techniques on a classification task on the publicly available dataset Camelyon17 and show that it is able to mitigate present batch effects.

Image and Video Processing,Computer Vision and Pattern Recognition
GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

Zhaoyang Xu,Xingru Huang,Carlos Fernández Moro,Béla Bozóky,Qianni Zhang

DOI: https://doi.org/10.48550/arXiv.1901.04059

2022-07-09

Abstract:Histopathological cancer diagnosis is based on visual examination of stained tissue slides. Hematoxylin and eosin (H\&E) is a standard stain routinely employed worldwide. It is easy to acquire and cost effective, but cells and tissue components show low-contrast with varying tones of dark blue and pink, which makes difficult visual assessments, digital image analysis, and quantifications. These limitations can be overcome by IHC staining of target proteins of the tissue slide. IHC provides a selective, high-contrast imaging of cells and tissue components, but their use is largely limited by a significantly more complex laboratory processing and high cost. We proposed a conditional CycleGAN (cCGAN) network to transform the H\&E stained images into IHC stained images, facilitating virtual IHC staining on the same slide. This data-driven method requires only a limited amount of labelled data but will generate pixel level segmentation results. The proposed cCGAN model improves the original network \cite{zhu_unpaired_2017} by adding category conditions and introducing two structural loss functions, which realize a multi-subdomain translation and improve the translation accuracy as well. % need to give reasons here. Experiments demonstrate that the proposed model outperforms the original method in unpaired image translation with multi-subdomains. We also explore the potential of unpaired images to image translation method applied on other histology images related tasks with different staining techniques.

Computer Vision and Pattern Recognition

Essays in exploration: new demographic history of the late 19th-century United States.

Dual contrastive learning based image-to-image translation of unstained skin tissue into virtually stained H&E images

Role of farnesoid X receptor (FXR) in the process of differentiation of bone marrow stromal cells into osteoblasts.

Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images

Using machine learning for chemical-free histological tissue staining

Single color digital H&E staining with In-and-Out Net

Deep learning-based transformation of the H&E stain into special stains

Virtual stain transfer in histology via cascaded deep neural networks

High Accuracy Tumor Diagnoses and Benchmarking of Hematoxylin and Eosin Stained Prostate Core Biopsy Images Generated by Explainable Deep Neural Networks

Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

Computational Histological Staining and Destaining of Prostate Core Biopsy RGB Images with Generative Adversarial Neural Networks

Stain Style Transfer of Histopathology Images Via Structure-Preserved Generative Learning

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

A Laplacian Pyramid Based Generative H&E Stain Augmentation Network

Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Unpaired Virtual Histological Staining Using Prior-Guided Generative Adversarial Networks.

Pix2Pix-based Stain-to-Stain Translation: A Solution for Robust Stain Normalization in Histopathology Images Analysis

StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining

Virtual multi-staining in a single-section view for renal pathology using generative adversarial networks

Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations

GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis