[Decrease of the noradrenaline level in asphyxial circulatory arrest in the dog].

Y. Blanloeil,J. Melchior,M. Pinaud

1985-06-01

Abstract:

What problem does this paper attempt to address?

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Shuting Liu,Baochang Zhang,Yiqing Liu,Anjia Han,Huijuan Shi,Tian Guan,Yonghong He

DOI: https://doi.org/10.1109/tmi.2021.3069874

IF: 10.6

2021-01-01

IEEE Transactions on Medical Imaging

Abstract:Pathological examination is the gold standard for the diagnosis of cancer. Common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry (IHC). In some cases, it is hard to make accurate diagnoses of cancer by referring only to H&E staining images. Whereas, the IHC examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual IHC images from H&E-stained images will be a good solution for current IHC examination hard accessibility issue, especially for some low-resource regions. However, existing approaches have limitations in microscopic structural preservation and the consistency of pathology properties. In addition, pixel-level paired data is hard available. In our work, we propose a novel adversarial learning method for effective Ki-67-stained image generation from corresponding H&E-stained image. Our method takes fully advantage of structural similarity constraint and skip connection to improve structural details preservation; and pathology consistency constraint and pathological representation network are first proposed to enforce the generated and source images hold the same pathological properties in different staining domains. We empirically demonstrate the effectiveness of our approach on two different unpaired histopathological datasets. Extensive experiments indicate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin. In addition, our approach also achieves a stable and good performance on unbalanced datasets, which shows our method has strong robustness. We believe that our method has significant potential in clinical virtual staining and advance the progress of computer-aided multi-staining histology image analysis.
GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

Zhaoyang Xu,Xingru Huang,Carlos Fernández Moro,Béla Bozóky,Qianni Zhang

DOI: https://doi.org/10.48550/arXiv.1901.04059

2022-07-09

Abstract:Histopathological cancer diagnosis is based on visual examination of stained tissue slides. Hematoxylin and eosin (H\&E) is a standard stain routinely employed worldwide. It is easy to acquire and cost effective, but cells and tissue components show low-contrast with varying tones of dark blue and pink, which makes difficult visual assessments, digital image analysis, and quantifications. These limitations can be overcome by IHC staining of target proteins of the tissue slide. IHC provides a selective, high-contrast imaging of cells and tissue components, but their use is largely limited by a significantly more complex laboratory processing and high cost. We proposed a conditional CycleGAN (cCGAN) network to transform the H\&E stained images into IHC stained images, facilitating virtual IHC staining on the same slide. This data-driven method requires only a limited amount of labelled data but will generate pixel level segmentation results. The proposed cCGAN model improves the original network \cite{zhu_unpaired_2017} by adding category conditions and introducing two structural loss functions, which realize a multi-subdomain translation and improve the translation accuracy as well. % need to give reasons here. Experiments demonstrate that the proposed model outperforms the original method in unpaired image translation with multi-subdomains. We also explore the potential of unpaired images to image translation method applied on other histology images related tasks with different staining techniques.

Computer Vision and Pattern Recognition
CytoGAN: Unpaired staining transfer by structure preservation for cytopathology image analysis

Ruijie Wang,Sicheng Yang,Qiling Li,Dexing Zhong

DOI: https://doi.org/10.1016/j.compbiomed.2024.108942

Abstract:With the development of digital pathology, deep learning is increasingly being applied to endometrial cell morphology analysis for cancer screening. And cytology images with different staining may degrade the performance of these analysis algorithms. To address the impact of staining patterns, many strategies have been proposed and hematoxylin and eosin (H&E) images have been transferred to other staining styles. However, none of the existing methods are able to generate realistic cytological images with preserved cellular layout, and many important clinical structural information is lost. To address the above issues, we propose a different staining transformation model, CytoGAN, which can quickly and realistically generate images with different staining styles. It includes a novel structure preservation module that preserves the cell structure well, even if the resolution or cell size between the source and target domains do not match. Meanwhile, a stain adaptive module is designed to help the model generate realistic and high-quality endometrial cytology images. We compared our model with ten state-of-the-art stain transformation models and evaluated by two pathologists. Furthermore, in the downstream endometrial cancer classification task, our algorithm improves the robustness of the classification model on multimodal datasets, with more than 20 % improvement in accuracy. We found that generating specified specific stains from existing H&E images improves the diagnosis of endometrial cancer. Our code will be available on github.
Virtual staining for mitosis detection in Breast Histopathology

Caner Mercan,Germonda Reijnen-Mooij,David Tellez Martin,Johannes Lotz,Nick Weiss,Marcel van Gerven,Francesco Ciompi

DOI: https://doi.org/10.48550/arXiv.2003.07801

2020-03-18

Abstract:We propose a virtual staining methodology based on Generative Adversarial Networks to map histopathology images of breast cancer tissue from H&E stain to PHH3 and vice versa. We use the resulting synthetic images to build Convolutional Neural Networks (CNN) for automatic detection of mitotic figures, a strong prognostic biomarker used in routine breast cancer diagnosis and grading. We propose several scenarios, in which CNN trained with synthetically generated histopathology images perform on par with or even better than the same baseline model trained with real images. We discuss the potential of this application to scale the number of training samples without the need for manual annotations.

Image and Video Processing,Computer Vision and Pattern Recognition
Virtual stain transfer in histology via cascaded deep neural networks

Xilin Yang,Bijie Bai,Yijie Zhang,Yuzhu Li,Kevin de Haan,Tairan Liu,Aydogan Ozcan

DOI: https://doi.org/10.1021/acsphotonics.2c00932

2022-07-14

Abstract:Pathological diagnosis relies on the visual inspection of histologically stained thin tissue specimens, where different types of stains are applied to bring contrast to and highlight various desired histological features. However, the destructive histochemical staining procedures are usually irreversible, making it very difficult to obtain multiple stains on the same tissue section. Here, we demonstrate a virtual stain transfer framework via a cascaded deep neural network (C-DNN) to digitally transform hematoxylin and eosin (H&E) stained tissue images into other types of histological stains. Unlike a single neural network structure which only takes one stain type as input to digitally output images of another stain type, C-DNN first uses virtual staining to transform autofluorescence microscopy images into H&E and then performs stain transfer from H&E to the domain of the other stain in a cascaded manner. This cascaded structure in the training phase allows the model to directly exploit histochemically stained image data on both H&E and the target special stain of interest. This advantage alleviates the challenge of paired data acquisition and improves the image quality and color accuracy of the virtual stain transfer from H&E to another stain. We validated the superior performance of this C-DNN approach using kidney needle core biopsy tissue sections and successfully transferred the H&E-stained tissue images into virtual PAS (periodic acid-Schiff) stain. This method provides high-quality virtual images of special stains using existing, histochemically stained slides and creates new opportunities in digital pathology by performing highly accurate stain-to-stain transformations.

Medical Physics,Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing,Quantitative Methods
Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images

Mehdi Ounissi,Ilias Sarbout,Jean-Pierre Hugot,Christine Martinez-Vinson,Dominique Berrebi,Daniel Racoceanu

2024-06-27

Abstract:Chemical staining methods are dependable but require extensive time, expensive chemicals, and raise environmental concerns. These challenges highlight the need for alternative solutions like virtual staining, which accelerates the diagnostic process and enhances stain application flexibility. Generative AI technologies are pivotal in addressing these issues. However, the high-stakes nature of healthcare decisions, especially in computational pathology, complicates the adoption of these tools due to their opaque processes. Our work introduces the use of generative AI for virtual staining, aiming to enhance performance, trustworthiness, scalability, and adaptability in computational pathology. The methodology centers on a singular H&E encoder supporting multiple stain decoders. This design focuses on critical regions in the latent space of H&E, enabling precise synthetic stain generation. Our method, tested to generate 8 different stains from a single H&E slide, offers scalability by loading only necessary model components during production. We integrate label-free knowledge in training, using loss functions and regularization to minimize artifacts, thus improving paired/unpaired virtual staining accuracy. To build trust, we use real-time self-inspection with discriminators for each stain type, providing pathologists with confidence heat-maps. Automatic quality checks on new H&E slides ensure conformity to the trained distribution, ensuring accurate synthetic stains. Recognizing pathologists' challenges with new technologies, we have developed an open-source, cloud-based system, that allows easy virtual staining of H&E slides through a browser, addressing hardware/software issues and facilitating real-time user feedback. We also curated a novel dataset of 8 paired H&E/stains related to pediatric Crohn's disease, comprising 480 WSIs to further stimulate computational pathology research.

Image and Video Processing,Computer Vision and Pattern Recognition
Virtual Immunohistochemistry Staining for Histological Images Assisted by Weakly-supervised Learning

Jiahan Li,Jiuyang Dong,Shenjin Huang,Xi Li,Junjun Jiang,Xiaopeng Fan,Yongbing Zhang

DOI: https://doi.org/10.1109/cvpr52733.2024.01070

2024-01-01

Abstract:Recently, virtual staining technology has greatly promoted the advancement of histopathology. Despite the practical successes achieved, the outstanding performance of most virtual staining methods relies on hard-to-obtain paired images in training. In this paper, we propose a method for virtual immunohistochemistry (IHC) staining, named confusion-GAN, which does not require paired images and can achieve comparable performance to supervised algorithms. Specifically, we propose a multi-branch discriminator, which judges if the features of generated images can be embedded into the feature pool of target domain images, to improve the visual quality of generated images. Meanwhile, we also propose a novel patch-level pathology information extractor, which is assisted by multiple instance learning, to ensure pathological consistency during virtual staining. Extensive experiments were conducted on three types of IHC images, including a high-resolution hepatocel-lular carcinoma immunohistochemical dataset proposed by us. The results demonstrated that our proposed confusion-GAN can generate highly realistic images that are capable of deceiving even experienced pathologists. Furthermore, compared to using H&E images directly, the downstream diagnosis achieved higher accuracy when using images generated by confusion-GAN. Our dataset and codes will be available at https://github.com/jiahanli2022/confusion-GAN.
Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

Christopher D. Walsh,Joanne Edwards,Robert H. Insall

DOI: https://doi.org/10.48550/arXiv.2204.06849

2022-04-14

Abstract:Immunohistochemistry is a valuable diagnostic tool for cancer pathology. However, it requires specialist labs and equipment, is time-intensive, and is difficult to reproduce. Consequently, a long term aim is to provide a digital method of recreating physical immunohistochemical stains. Generative Adversarial Networks have become exceedingly advanced at mapping one image type to another and have shown promise at inferring immunostains from haematoxylin and eosin. However, they have a substantial weakness when used with pathology images as they can fabricate structures that are not present in the original data. CycleGANs can mitigate invented tissue structures in pathology image mapping but have a related disposition to generate areas of inaccurate staining. In this paper, we describe a modification to the loss function of a CycleGAN to improve its mapping ability for pathology images by enforcing realistic stain replication while retaining tissue structure. Our approach improves upon others by considering structure and staining during model training. We evaluated our network using the Fréchet Inception distance, coupled with a new technique that we propose to appraise the accuracy of virtual immunohistochemistry. This assesses the overlap between each stain component in the inferred and ground truth images through colour deconvolution, thresholding and the Sorensen-Dice coefficient. Our modified loss function resulted in a Dice coefficient for the virtual stain of 0.78 compared with the real AE1/AE3 slide. This was superior to the unaltered CycleGAN's score of 0.74. Additionally, our loss function improved the Fréchet Inception distance for the reconstruction to 74.54 from 76.47. We, therefore, describe an advance in virtual restaining that can extend to other immunostains and tumour types and deliver reproducible, fast and readily accessible immunohistochemistry worldwide.

Image and Video Processing,Computer Vision and Pattern Recognition,Quantitative Methods
Structural Cycle GAN for Virtual Immunohistochemistry Staining of Gland Markers in the Colon

Shikha Dubey,Tushar Kataria,Beatrice Knudsen,Shireen Y. Elhabian

2023-08-25

Abstract:With the advent of digital scanners and deep learning, diagnostic operations may move from a microscope to a desktop. Hematoxylin and Eosin (H&E) staining is one of the most frequently used stains for disease analysis, diagnosis, and grading, but pathologists do need different immunohistochemical (IHC) stains to analyze specific structures or cells. Obtaining all of these stains (H&E and different IHCs) on a single specimen is a tedious and time-consuming task. Consequently, virtual staining has emerged as an essential research direction. Here, we propose a novel generative model, Structural Cycle-GAN (SC-GAN), for synthesizing IHC stains from H&E images, and vice versa. Our method expressly incorporates structural information in the form of edges (in addition to color data) and employs attention modules exclusively in the decoder of the proposed generator model. This integration enhances feature localization and preserves contextual information during the generation process. In addition, a structural loss is incorporated to ensure accurate structure alignment between the generated and input markers. To demonstrate the efficacy of the proposed model, experiments are conducted with two IHC markers emphasizing distinct structures of glands in the colon: the nucleus of epithelial cells (CDX2) and the cytoplasm (CK818). Quantitative metrics such as FID and SSIM are frequently used for the analysis of generative models, but they do not correlate explicitly with higher-quality virtual staining results. Therefore, we propose two new quantitative metrics that correlate directly with the virtual staining specificity of IHC markers.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning,Quantitative Methods
Revealing 3D cancer tissue structures using holotomography and virtual hematoxylin and eosin staining via deep learning

Juyeon Park,Su-Jin Shin,Minji Kim,Geon Kim,Hyungjoo Cho,Dongmin Ryu,Daewoong Ahn,Ji Eun Heo,Inyeop Jang,Hyun-seok Min,Kwang Suk Lee,Tae Hyun Hwang,YongKeun Park

DOI: https://doi.org/10.1101/2023.12.04.569853

2024-02-19

Abstract:In standard histopathology, hematoxylin and eosin (H&E) staining stands as a pivotal tool for cancer tissue analysis. However, this method is limited to two-dimensional (2D) analysis or requires labor-intensive preparation for three-dimensional (3D) inspection of cancer tissues. In this study, we present a method for 3D virtual H&E staining of label-free cancer tissues, employing holotomography and deep learning. Holotomography is used to measure the 3D refractive index (RI) distribution of the label-free cancer slides. A deep learning-based image-to-image translation framework is integrated into the resulting 3D RI distribution, enabling virtual H&E staining in 3D. Our method has been applied to colon cancer tissue slides with thicknesses up to 20 μm, with conventional chemical H&E staining providing a direct validation for the method. This framework not only bypasses the conventional staining process but also provides 3D structures of glands, lumens, and individual nuclei. The results demonstrate enhancement in histopathological efficiency and the extension of the standard histopathology into the 3D realm. To validate the repeatability and scalability of the approach, we applied the framework to the gastric cancer slides obtained from different institute and imaging devices.

Cancer Biology
Determination of malignancy of thyroid nodules with positron emission tomography.

Allen D. Bloom,L. Adler,J. Shuck

IF: 4.348

1993-10-01

Surgery

Abstract:BACKGROUND Current noninvasive evaluation techniques of thyroid nodules are nonspecific. We studied the use of a new noninvasive imaging modality, positron emission tomography (PET), in the evaluation of thyroid nodules. METHODS Nineteen patients were studied before operation with PET with [18F]-2-deoxy-2-fluoro-D-glucose (FDG). Twelve patients had solitary thyroid nodules and seven had multinodular goiters. Fourteen overlapping transverse slices were acquired through the thyroid gland. A region of interest was drawn around the abnormal area and the maximum pixel value was calculated and corrected to produce a normalized value for FDG uptake. RESULTS Four of the 12 nodules were malignant (three papillary and one follicular carcinoma), with the others being follicular adenomas. All of the multinodular goiters were benign. All of the malignancies had an FDG greater than 8.5 and all of the benign nodules had an FDG less than 7.6. The dose uptake ratio of FDG was 10.8 +/- 3.2 (mean +/- SD) for the malignancies and was significantly greater than the dose uptake ratio of either the follicular adenomas (4.3 +/- 2.0) or the dominant nodule in multinodular goiters (3.0 +/- 2.0). CONCLUSIONS The FDG uptake as measured by PET scanning successfully discriminated between all benign and malignant tumors. This technique appears useful in the evaluation of thyroid nodules.
High Accuracy Tumor Diagnoses and Benchmarking of Hematoxylin and Eosin Stained Prostate Core Biopsy Images Generated by Explainable Deep Neural Networks

Aman Rana,Alarice Lowe,Marie Lithgow,Katharine Horback,Tyler Janovitz,Annacarolina Da Silva,Harrison Tsai,Vignesh Shanmugam,Hyung-Jin Yoon,Pratik Shah

DOI: https://doi.org/10.1001/jamanetworkopen.2020.5111

2019-08-02

Abstract:Histopathological diagnoses of tumors in tissue biopsy after Hematoxylin and Eosin (H&E) staining is the gold standard for oncology care. H&E staining is slow and uses dyes, reagents and precious tissue samples that cannot be reused. Thousands of native nonstained RGB Whole Slide Image (RWSI) patches of prostate core tissue biopsies were registered with their H&E stained versions. Conditional Generative Adversarial Neural Networks (cGANs) that automate conversion of native nonstained RWSI to computational H&E stained images were then trained. High similarities between computational and H&E dye stained images with Structural Similarity Index (SSIM) 0.902, Pearsons Correlation Coefficient (CC) 0.962 and Peak Signal to Noise Ratio (PSNR) 22.821 dB were calculated. A second cGAN performed accurate computational destaining of H&E dye stained images back to their native nonstained form with SSIM 0.9, CC 0.963 and PSNR 25.646 dB. A single-blind study computed more than 95% pixel-by-pixel overlap between prostate tumor annotations on computationally stained images, provided by five-board certified MD pathologists, with those on H&E dye stained counterparts. We report the first visualization and explanation of neural network kernel activation maps during H&E staining and destaining of RGB images by cGANs. High similarities between kernel activation maps of computational and H&E stained images (Mean-Squared Errors <0.0005) provide additional mathematical and mechanistic validation of the staining system. Our neural network framework thus is automated, explainable and performs high precision H&E staining and destaining of low cost native RGB images, and is computer vision and physician authenticated for rapid and accurate tumor diagnoses.

Quantitative Methods,Computer Vision and Pattern Recognition,Image and Video Processing
Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

Bijie Bai,Hongda Wang,Yuzhu Li,Kevin de Haan,Francesco Colonnese,Yujie Wan,Jingyi Zuo,Ngan B. Doan,Xiaoran Zhang,Yijie Zhang,Jingxi Li,Xilin Yang,Wenjie Dong,Morgan Angus Darrow,Elham Kamangar,Han Sung Lee,Yair Rivenson,Aydogan Ozcan

DOI: https://doi.org/10.34133/2022/9786242

2022-10-27

BME Frontiers

Abstract:The immunohistochemical (IHC) staining of the human epidermal growth factor receptor 2 (HER2) biomarker is widely practiced in breast tissue analysis, preclinical studies, and diagnostic decisions, guiding cancer treatment and investigation of pathogenesis. HER2 staining demands laborious tissue treatment and chemical processing performed by a histotechnologist, which typically takes one day to prepare in a laboratory, increasing analysis time and associated costs. Here, we describe a deep learning-based virtual HER2 IHC staining method using a conditional generative adversarial network that is trained to rapidly transform autofluorescence microscopic images of unlabeled/label-free breast tissue sections into bright-field equivalent microscopic images, matching the standard HER2 IHC staining that is chemically performed on the same tissue sections. The efficacy of this virtual HER2 staining framework was demonstrated by quantitative analysis, in which three board-certified breast pathologists blindly graded the HER2 scores of virtually stained and immunohistochemically stained HER2 whole slide images (WSIs) to reveal that the HER2 scores determined by inspecting virtual IHC images are as accurate as their immunohistochemically stained counterparts. A second quantitative blinded study performed by the same diagnosticians further revealed that the virtually stained HER2 images exhibit a comparable staining quality in the level of nuclear detail, membrane clearness, and absence of staining artifacts with respect to their immunohistochemically stained counterparts. This virtual HER2 staining framework bypasses the costly, laborious, and time-consuming IHC staining procedures in laboratory and can be extended to other types of biomarkers to accelerate the IHC tissue staining used in life sciences and biomedical workflow.

English Else
Virtual Histology with Photon Absorption Remote Sensing using a Cycle-Consistent Generative Adversarial Network with Weakly Registered Pairs

James E. D. Tweel,Benjamin R. Ecclestone,Marian Boktor,James Alexander Tummon Simmons,Paul Fieguth,Parsin Haji Reza

2023-06-27

Abstract:Modern histopathology relies on the microscopic examination of thin tissue sections stained with histochemical techniques, typically using brightfield or fluorescence microscopy. However, the staining of samples can permanently alter their chemistry and structure, meaning an individual tissue section must be prepared for each desired staining contrast. This not only consumes valuable tissue samples but also introduces delays in essential diagnostic timelines. In this work, virtual histochemical staining is developed using label-free photon absorption remote sensing (PARS) microscopy. We present a method that generates virtually stained histology images that are indistinguishable from the gold standard hematoxylin and eosin (H&E) staining. First, PARS label-free ultraviolet absorption images are captured directly within unstained tissue specimens. The radiative and non-radiative absorption images are then preprocessed, and virtually stained through the presented pathway. The preprocessing pipeline features a self-supervised Noise2Void denoising convolutional neural network (CNN) as well as a novel algorithm for pixel-level mechanical scanning error correction. These developments significantly enhance the recovery of sub-micron tissue structures, such as nucleoli location and chromatin distribution. Finally, we used a cycle-consistent generative adversarial network CycleGAN architecture to virtually stain the preprocessed PARS data. Virtual staining is applied to thin unstained sections of malignant human skin and breast tissue samples. Clinically relevant details are revealed, with comparable contrast and quality to gold standard H&E-stained images. This work represents a crucial step to deploying label-free microscopy as an alternative to standard histopathology techniques.

Medical Physics,Image and Video Processing
Bi-directional Feature Fusion Generative Adversarial Network for Ultra-high Resolution Pathological Image Virtual Re-staining.

Kexin Sun,Zhineng Chen,Gongwei Wang,Jun Liu,Xiongjun Ye,Yu-Gang Jiang

DOI: https://doi.org/10.1109/cvpr52729.2023.00380

2023-01-01

Abstract:The cost of pathological examination makes virtual restaining of pathological images meaningful. However, due to the ultra-high resolution of pathological images, traditional virtual restaining methods have to divide a WSI image into patches for model training and inference. Such a limitation leads to the lack of global information, resulting in observable differences in color, brightness and contrast when the re-stained patches are merged to generate an image of larger size. We summarize this issue as the square effect. Some existing methods try to solve this issue through overlapping between patches or simple postprocessing. But the former one is not that effective, while the latter one requires carefully tuning. In order to eliminate the square effect, we design a bidirectional feature fusion generative adversarial network (BFF-GAN) with a global branch and a local branch. It learns the inter-patch connections through the fusion of global and local features plus patch-wise attention. We perform experiments on both the private dataset RCC and the public dataset ANHIR. The results show that our model achieves competitive performance and is able to generate extremely real images that are deceptive even for experienced pathologists, which means it is of great clinical significance.
Multi-Channel Feature Extraction for Virtual Histological Staining of Photon Absorption Remote Sensing Images

Marian Boktor,James E.D. Tweel,Benjamin R. Ecclestone,Jennifer Ai Ye,Paul Fieguth,Parsin Haji Reza

2023-07-05

Abstract:Accurate and fast histological staining is crucial in histopathology, impacting diagnostic precision and reliability. Traditional staining methods are time-consuming and subjective, causing delays in diagnosis. Digital pathology plays a vital role in advancing and optimizing histology processes to improve efficiency and reduce turnaround times. This study introduces a novel deep learning-based framework for virtual histological staining using photon absorption remote sensing (PARS) images. By extracting features from PARS time-resolved signals using a variant of the K-means method, valuable multi-modal information is captured. The proposed multi-channel cycleGAN (MC-GAN) model expands on the traditional cycleGAN framework, allowing the inclusion of additional features. Experimental results reveal that specific combinations of features outperform the conventional channels by improving the labeling of tissue structures prior to model training. Applied to human skin and mouse brain tissue, the results underscore the significance of choosing the optimal combination of features, as it reveals a substantial visual and quantitative concurrence between the virtually stained and the gold standard chemically stained hematoxylin and eosin (H&E) images, surpassing the performance of other feature combinations. Accurate virtual staining is valuable for reliable diagnostic information, aiding pathologists in disease classification, grading, and treatment planning. This study aims to advance label-free histological imaging and opens doors for intraoperative microscopy applications.

Image and Video Processing,Medical Physics
Generative Adversarial Networks Accurately Reconstruct Pan-Cancer Histology from Pathologic, Genomic, and Radiographic Latent Features

Frederick M. Howard,Hanna M. Hieromnimon,Siddhi Ramesh,James Dolezal,Sara Kochanny,Qianchen Zhang,Brad Feiger,Joseph Peterson,Cheng Fan,Charles M. Perou,Jasmine Vickery,Megan Sullivan,Kimberly Cole,Galina Khramtsova,Alexander T. Pearson

DOI: https://doi.org/10.1101/2024.03.22.586306

2024-03-25

Abstract:Artificial intelligence models have been increasingly used in the analysis of tumor histology to perform tasks ranging from routine classification to identification of novel molecular features. These approaches distill cancer histologic images into high-level features which are used in predictions, but understanding the biologic meaning of such features remains challenging. We present and validate a custom generative adversarial network – HistoXGAN – capable of reconstructing representative histology using feature vectors produced by common feature extractors. We evaluate HistoXGAN across 29 cancer subtypes and demonstrate that reconstructed images retain information regarding tumor grade, histologic subtype, and gene expression patterns. We leverage HistoXGAN to illustrate the underlying histologic features for deep learning models for actionable mutations, identify model reliance on histologic batch effect in predictions, and demonstrate accurate reconstruction of tumor histology from radiographic imaging for a ‘virtual biopsy’.

Cancer Biology
Versatile Stain Transfer in Histopathology Using a Unified Diffusion Framework

Xudong Yan,Mingze Yuan,Yao Lu,Ying Zhang,Zifan Chen,Peng Bao,Zehua Li,Bin Dong,Li Yang,Li Zhang,Fangxu Zhou

DOI: https://doi.org/10.1101/2024.11.23.624680

2024-11-23

Abstract:Histological staining is vital in clinical pathology for visualizing tissue structures. However, these techniques are laborious and time-consuming. Digital virtual staining offers a promising solution, but existing methods typically rely on Generative Adversarial Networks (GANs), which may suffer from artifacts and mode collapse. Motivated by the success of diffusion models, we present DUST, a novel Diffusion-based Unified framework for versatile Stain Transfer in histopathology. To enhance domain awareness and task-specific performance, we propose a dual encoding strategy that integrates the stain types of both the source and target domains. Additionally, we introduce a dynamic dual-output head to address the unstable intensity issue encountered with conventional DDPM implementations. Validated on a curated four-stain kidney histopathological dataset (H&E, MT, PAS, and PASM), DUST demonstrates superior versatile stain transfer capabilities. Our research highlights the potential of diffusion models to advance virtual staining, paving the way for more efficient digital pathology analyses.

Biology
Structurally constrained and pathology-aware convolutional transformer generative adversarial network for virtual histology staining of human coronary optical coherence tomography images

Xueshen Li,Hongshan Liu,Xiaoyu Song,Charles C. Marboe,Brigitta C. Brott,Silvio H. Litovsky,Yu Gan

DOI: https://doi.org/10.1117/1.jbo.29.3.036004

IF: 3.758

2024-03-27

Journal of Biomedical Optics

Abstract:SignificanceThere is a significant need for the generation of virtual histological information from coronary optical coherence tomography (OCT) images to better guide the treatment of coronary artery disease (CAD). However, existing methods either require a large pixel-wise paired training dataset or have limited capability to map pathological regions.AimThe aim of this work is to generate virtual histological information from coronary OCT images, without a pixel-wise paired training dataset while capable of providing pathological patterns.ApproachWe design a structurally constrained, pathology-aware, transformer generative adversarial network, namely structurally constrained pathology-aware convolutional transformer generative adversarial network (SCPAT-GAN), to generate virtual stained H&E histology from OCT images. We quantitatively evaluate the quality of virtual stained histology images by measuring the Fréchet inception distance (FID) and perceptual hash value (PHV). Moreover, we invite experienced pathologists to evaluate the virtual stained images. Furthermore, we visually inspect the virtual stained image generated by SCPAT-GAN. Also, we perform an ablation study to validate the design of the proposed SCPAT-GAN. Finally, we demonstrate 3D virtual stained histology images.ResultsCompared to previous research, the proposed SCPAT-GAN achieves better FID and PHV scores. The visual inspection suggests that the virtual histology images generated by SCPAT-GAN resemble both normal and pathological features without artifacts. As confirmed by the pathologists, the virtual stained images have good quality compared to real histology images. The ablation study confirms the effectiveness of the combination of proposed pathological awareness and structural constraining modules.ConclusionsThe proposed SCPAT-GAN is the first to demonstrate the feasibility of generating both normal and pathological patterns without pixel-wisely supervised training. We expect the SCPAT-GAN to assist in the clinical evaluation of treating the CAD by providing 2D and 3D histopathological visualizations.

radiology, nuclear medicine & medical imaging,biochemical research methods,optics
Computational staining of pathology images to study tumor microenvironment in lung cancer

Shidan Wang,Ruichen Rong,Donghan M. Yang,Ling Cai,Lin Yang,Danni Luo,Bo Yao,Lin Xu,Tao Wang,Xiaowei Zhan,Yang Xie,Adi Gazdar,John Minna,Guanghua Xiao

DOI: https://doi.org/10.1101/630749

2019-05-07

Abstract:ABSTRACT The spatial organization of different types of cells in tumor tissues reveals important information about the tumor microenvironment (TME). In order to facilitate the study of cellular spatial organization and interactions, we developed a comprehensive nuclei segmentation and classification tool to characterize the TME from standard Hematoxylin and Eosin (H&E)-stained pathology images. This tool can computationally “stain” different types of cell nuclei in H&E pathology images to facilitate pathologists in analyzing the TME. A Mask Regional-Convolutional Neural Network (Mask-RCNN) model was developed to segment the nuclei of tumor, stromal, lymphocyte, macrophage, karyorrhexis and red blood cells in lung adenocarcinoma (ADC). Using this tool, we identified and classified cell nuclei and extracted 48 cell spatial organization-related features that characterize the TME. Using these features, we developed a prognostic model from the National Lung Screening Trial dataset, and independently validated the model in The Cancer Genome Atlas (TCGA) lung ADC dataset, in which the predicted high-risk group showed significantly worse survival than the low-risk group (pv= 0.001), with a hazard ratio of 2.23 [1.37-3.65] after adjusting for clinical variables. Furthermore, the image-derived TME features were significantly correlated with the gene expression of biological pathways. For example, transcription activation of both the T-cell receptor (TCR) and Programmed cell death protein 1 (PD1) pathways was positively correlated with the density of detected lymphocytes in tumor tissues, while expression of the extracellular matrix organization pathway was positively correlated with the density of stromal cells. This study developed a deep learning-based analysis tool to dissect the TME from tumor tissue images. Using this tool, we demonstrated that the spatial organization of different cell types is predictive of patient survival and associated with the gene expression of biological pathways. Although developed from the pathology images of lung ADC, this model can be adapted into other types of cancers.

[Decrease of the noradrenaline level in asphyxial circulatory arrest in the dog].

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

CytoGAN: Unpaired staining transfer by structure preservation for cytopathology image analysis

Virtual staining for mitosis detection in Breast Histopathology

Virtual stain transfer in histology via cascaded deep neural networks

Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images

Virtual Immunohistochemistry Staining for Histological Images Assisted by Weakly-supervised Learning

Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

Structural Cycle GAN for Virtual Immunohistochemistry Staining of Gland Markers in the Colon

Revealing 3D cancer tissue structures using holotomography and virtual hematoxylin and eosin staining via deep learning

Determination of malignancy of thyroid nodules with positron emission tomography.

High Accuracy Tumor Diagnoses and Benchmarking of Hematoxylin and Eosin Stained Prostate Core Biopsy Images Generated by Explainable Deep Neural Networks

Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

Virtual Histology with Photon Absorption Remote Sensing using a Cycle-Consistent Generative Adversarial Network with Weakly Registered Pairs

Bi-directional Feature Fusion Generative Adversarial Network for Ultra-high Resolution Pathological Image Virtual Re-staining.

Multi-Channel Feature Extraction for Virtual Histological Staining of Photon Absorption Remote Sensing Images

Generative Adversarial Networks Accurately Reconstruct Pan-Cancer Histology from Pathologic, Genomic, and Radiographic Latent Features

Versatile Stain Transfer in Histopathology Using a Unified Diffusion Framework

Structurally constrained and pathology-aware convolutional transformer generative adversarial network for virtual histology staining of human coronary optical coherence tomography images

Computational staining of pathology images to study tumor microenvironment in lung cancer