Abstract:The lack of large datasets and high-quality annotated data often limits the development of accurate and robust machine-learning models within the medical and surgical domains. In the machine learning community, generative models have recently demonstrated that it is possible to produce novel and diverse synthetic images that closely resemble reality while controlling their content with various types of annotations. However, generative models have not been yet fully explored in the surgical domain, partially due to the lack of large datasets and due to specific challenges present in the surgical domain such as the large anatomical diversity. We propose Surgery-GAN, a novel generative model that produces synthetic images from segmentation maps. Our architecture produces surgical images with improved quality when compared to early generative models thanks to the combination of channel- and pixel-level normalization layers that boost image quality while granting adherence to the input segmentation map. While state-of-the-art generative models often generate overfitted images, lacking diversity, or containing unrealistic artefacts such as cartooning; experiments demonstrate that Surgery-GAN is able to generate novel, realistic, and diverse surgical images in three different surgical datasets: cholecystectomy, partial nephrectomy, and radical prostatectomy. In addition, we investigate whether the use of synthetic images together with real ones can be used to improve the performance of other machine-learning models. Specifically, we use Surgery-GAN to generate large synthetic datasets which we then use to train five different segmentation models. Results demonstrate that using our synthetic images always improves the mean segmentation performance with respect to only using real images. For example, when considering radical prostatectomy, we can boost the mean segmentation performance by up to 5.43%. More interestingly, experimental results indicate that the performance improvement is larger in the set of classes that are under-represented in the training sets, where the performance boost of specific classes reaches up to 61.6%.

Synthesizing brain tumor images and annotations by combining progressive growing GAN and SPADE

Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection

Brain tumor segmentation using synthetic MR images -- A comparison of GANs and diffusion models

Synthesis of realistic fetal MRI with conditional Generative Adversarial Networks

Improvement of Multiparametric MR Image Segmentation by Augmenting the Data With Generative Adversarial Networks for Glioma Patients

Medical image synthesis via conditional GANs: Application to segmenting brain tumours

Brain tumor image generation using an aggregation of GAN models with style transfer

Lesion Mask-based Simultaneous Synthesis of Anatomic and MolecularMR Images using a GAN

Multi-modal brain tumor segmentation via conditional synthesis with Fourier domain adaptation

GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks

Generating multi-pathological and multi-modal images and labels for brain MRI

Using a generative adversarial network to generate synthetic MRI images for multi-class automatic segmentation of brain tumors

Nimg-65. Synthetic Mri Generation To Facilitate Autosegmentation Of Pediatric Brain Tumors In Limited Data Scenarios

Evaluating synthetic neuroimaging data augmentation for automatic brain tumour segmentation with a deep fully-convolutional network

sTBI-GAN: An adversarial learning approach for data synthesis on traumatic brain segmentation

Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans

Synthetic augmentation for semantic segmentation of class imbalanced biomedical images: A data pair generative adversarial network approach

Guided image generation for improved surgical image segmentation

TBI-GAN: An Adversarial Learning Approach for Data Synthesis on Traumatic Brain Segmentation

Label-informed cardiac magnetic resonance image synthesis through conditional generative adversarial networks

High-resolution medical image synthesis using progressively grown generative adversarial networks