Abstract:Synthesis of a large set of high-quality medical images with variability in anatomical representation and image appearance has the potential to provide solutions for tackling the scarcity of properly annotated data in medical image analysis research. In this paper, we propose a novel framework consisting of image segmentation and synthesis based on mask-conditional GANs for generating high-fidelity and diverse Cardiac Magnetic Resonance (CMR) images. The framework consists of two modules: i) a segmentation module trained using a physics-based simulated database of CMR images to provide multi-tissue labels on real CMR images, and ii) a synthesis module trained using pairs of real CMR images and corresponding multi-tissue labels, to translate input segmentation masks to realistic-looking cardiac images. The anatomy of synthesized images is based on labels, whereas the appearance is learned from the training images. We investigate the effects of the number of tissue labels, quantity of training data, and multi-vendor data on the quality of the synthesized images. Furthermore, we evaluate the effectiveness and usability of the synthetic data for a downstream task of training a deep-learning model for cardiac cavity segmentation in the scenarios of data replacement and augmentation. The results of the replacement study indicate that segmentation models trained with only synthetic data can achieve comparable performance to the baseline model trained with real data, indicating that the synthetic data captures the essential characteristics of its real counterpart. Furthermore, we demonstrate that augmenting real with synthetic data during training can significantly improve both the Dice score (maximum increase of 4%) and Hausdorff Distance (maximum reduction of 40%) for cavity segmentation, suggesting a good potential to aid in tackling medical data scarcity.

Shape-Consistent Generative Adversarial Networks for Multi-Modal Medical Segmentation Maps

Towards cross-modal organ translation and segmentation: A cycle- and shape-consistent generative adversarial network

Combining generative modelling and semi-supervised domain adaptation for whole heart cardiovascular magnetic resonance angiography segmentation

Unified Generative Adversarial Networks for Multimodal Segmentation from Unpaired 3D Medical Images.

Multi-Task Generative Adversarial Network for Handling Imbalanced Clinical Data

Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization

Label-informed cardiac magnetic resonance image synthesis through conditional generative adversarial networks

Semi-supervised Learning Approach to Generate Neuroimaging Modalities with Adversarial Training

SUSAN: Segment Unannotated image Structure using Adversarial Network

Reducing segmentation failures in cardiac MRI via Late feature fusion and GAN-based augmentation

Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning

DualMMP-GAN: Dual-scale Multi-Modality Perceptual Generative Adversarial Network for Medical Image Segmentation

Joint Semi-supervised 3D Super-Resolution and Segmentation with Mixed Adversarial Gaussian Domain Adaptation

SA-GAN: Structure-Aware GAN for Organ-Preserving Synthetic CT Generation

Generating multi-pathological and multi-modal images and labels for brain MRI

Generative Adversarial Networks for Pre-training of Medical Image Segmentation Networks

Generalisable Cardiac Structure Segmentation via Attentional and Stacked Image Adaptation

Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis

Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

Multi-Model Medical Image Segmentation Using Multi-Stage Generative Adversarial Network

Diverse Data Augmentation for Learning Image Segmentation with Cross-Modality Annotations.