Abstract:Synthesis of a large set of high-quality medical images with variability in anatomical representation and image appearance has the potential to provide solutions for tackling the scarcity of properly annotated data in medical image analysis research. In this paper, we propose a novel framework consisting of image segmentation and synthesis based on mask-conditional GANs for generating high-fidelity and diverse Cardiac Magnetic Resonance (CMR) images. The framework consists of two modules: i) a segmentation module trained using a physics-based simulated database of CMR images to provide multi-tissue labels on real CMR images, and ii) a synthesis module trained using pairs of real CMR images and corresponding multi-tissue labels, to translate input segmentation masks to realistic-looking cardiac images. The anatomy of synthesized images is based on labels, whereas the appearance is learned from the training images. We investigate the effects of the number of tissue labels, quantity of training data, and multi-vendor data on the quality of the synthesized images. Furthermore, we evaluate the effectiveness and usability of the synthetic data for a downstream task of training a deep-learning model for cardiac cavity segmentation in the scenarios of data replacement and augmentation. The results of the replacement study indicate that segmentation models trained with only synthetic data can achieve comparable performance to the baseline model trained with real data, indicating that the synthetic data captures the essential characteristics of its real counterpart. Furthermore, we demonstrate that augmenting real with synthetic data during training can significantly improve both the Dice score (maximum increase of 4%) and Hausdorff Distance (maximum reduction of 40%) for cavity segmentation, suggesting a good potential to aid in tackling medical data scarcity.

Label-informed cardiac magnetic resonance image synthesis through conditional generative adversarial networks

XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms

OA-GAN: Organ-Aware Generative Adversarial Network for Synthesizing Contrast-Enhanced Medical Images

Medical Image Synthesis with Generative Adversarial Networks for Tissue Recognition

Optimization of Cardiac Magnetic Resonance Synthetic Image Based on Simulated Generative Adversarial Network

Lesion Mask-based Simultaneous Synthesis of Anatomic and MolecularMR Images using a GAN

Less is More: Unsupervised Mask-guided Annotated CT Image Synthesis with Minimum Manual Segmentations

Controllable cardiac synthesis via disentangled anatomy arithmetic

Arterial Spin Labeling Image Synthesis from Structural MRI Using Improved Capsule-Based Networks

A Data Augmentation Pipeline to Generate Synthetic Labeled Datasets of 3D Echocardiography Images using a GAN

Shape-Consistent Generative Adversarial Networks for Multi-Modal Medical Segmentation Maps

An Attentive-based Generative Model for Medical Image Synthesis

End-to-End 3D Liver CT Image Synthesis from Vasculature Using a Multi-Task Conditional Generative Adversarial Network

Reducing segmentation failures in cardiac MRI via Late feature fusion and GAN-based augmentation

Red-GAN: Attacking Class Imbalance Via Conditioned Generation. Yet Another Perspective on Medical Image Synthesis for Skin Lesion Dermoscopy and Brain Tumor MRI

Towards cross-modal organ translation and segmentation: A cycle- and shape-consistent generative adversarial network

Synthesis of Registered Multimodal Medical Images with Lesions.

Synthetic Medical Images from Dual Generative Adversarial Networks

Synthetic Brain Images: Bridging the Gap in Brain Mapping With Generative Adversarial Model

SA-GAN: Structure-Aware GAN for Organ-Preserving Synthetic CT Generation

Multimodal MR Image Synthesis Using Gradient Prior and Adversarial Learning