Abstract:SignificanceThere is a significant need for the generation of virtual histological information from coronary optical coherence tomography (OCT) images to better guide the treatment of coronary artery disease (CAD). However, existing methods either require a large pixel-wise paired training dataset or have limited capability to map pathological regions.AimThe aim of this work is to generate virtual histological information from coronary OCT images, without a pixel-wise paired training dataset while capable of providing pathological patterns.ApproachWe design a structurally constrained, pathology-aware, transformer generative adversarial network, namely structurally constrained pathology-aware convolutional transformer generative adversarial network (SCPAT-GAN), to generate virtual stained H&E histology from OCT images. We quantitatively evaluate the quality of virtual stained histology images by measuring the Fréchet inception distance (FID) and perceptual hash value (PHV). Moreover, we invite experienced pathologists to evaluate the virtual stained images. Furthermore, we visually inspect the virtual stained image generated by SCPAT-GAN. Also, we perform an ablation study to validate the design of the proposed SCPAT-GAN. Finally, we demonstrate 3D virtual stained histology images.ResultsCompared to previous research, the proposed SCPAT-GAN achieves better FID and PHV scores. The visual inspection suggests that the virtual histology images generated by SCPAT-GAN resemble both normal and pathological features without artifacts. As confirmed by the pathologists, the virtual stained images have good quality compared to real histology images. The ablation study confirms the effectiveness of the combination of proposed pathological awareness and structural constraining modules.ConclusionsThe proposed SCPAT-GAN is the first to demonstrate the feasibility of generating both normal and pathological patterns without pixel-wisely supervised training. We expect the SCPAT-GAN to assist in the clinical evaluation of treating the CAD by providing 2D and 3D histopathological visualizations.

SCPAT-GAN: Structural Constrained and Pathology Aware Convolutional Transformer-GAN for Virtual Histology Staining of Human Coronary OCT images

Structurally constrained and pathology-aware convolutional transformer generative adversarial network for virtual histology staining of human coronary optical coherence tomography images

Structural constrained virtual histology staining for human coronary imaging using deep learning

TcGAN: Semantic-Aware and Structure-Preserved GANs with Individual Vision Transformer for Fast Arbitrary One-Shot Image Generation

Synthesizing Contrast-enhanced Computed Tomography Images with an Improved Conditional Generative Adversarial Network

Conditioned Generative Transformers for Histopathology Image Synthetic Augmentation

GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification

Structural Cycle GAN for Virtual Immunohistochemistry Staining of Gland Markers in the Colon

MC-GAT: multi-layer collaborative generative adversarial transformer for cholangiocarcinoma classification from hyperspectral pathological images

CytoGAN: Unpaired staining transfer by structure preservation for cytopathology image analysis

SA-GAN: Structure-Aware GAN for Organ-Preserving Synthetic CT Generation

Stain Style Transfer of Histopathology Images Via Structure-Preserved Generative Learning

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Unified Framework for Histopathology Image Augmentation and Classification via Generative Models

TCGAN: a transformer-enhanced GAN for PET synthetic CT

Virtual multi-staining in a single-section view for renal pathology using generative adversarial networks

AV-GAN: Attention-Based Varifocal Generative Adversarial Network for Uneven Medical Image Translation

Effective Immunohistochemistry Pathology Microscopy Image Generation Using CycleGAN

Virtual Immunohistochemistry Staining for Histological Images Assisted by Weakly-supervised Learning

CT-GAN: A conditional Generative Adversarial Network of transformer architecture for text-to-image