Abstract:The lack of large datasets and high-quality annotated data often limits the development of accurate and robust machine-learning models within the medical and surgical domains. In the machine learning community, generative models have recently demonstrated that it is possible to produce novel and diverse synthetic images that closely resemble reality while controlling their content with various types of annotations. However, generative models have not been yet fully explored in the surgical domain, partially due to the lack of large datasets and due to specific challenges present in the surgical domain such as the large anatomical diversity. We propose Surgery-GAN, a novel generative model that produces synthetic images from segmentation maps. Our architecture produces surgical images with improved quality when compared to early generative models thanks to the combination of channel- and pixel-level normalization layers that boost image quality while granting adherence to the input segmentation map. While state-of-the-art generative models often generate overfitted images, lacking diversity, or containing unrealistic artefacts such as cartooning; experiments demonstrate that Surgery-GAN is able to generate novel, realistic, and diverse surgical images in three different surgical datasets: cholecystectomy, partial nephrectomy, and radical prostatectomy. In addition, we investigate whether the use of synthetic images together with real ones can be used to improve the performance of other machine-learning models. Specifically, we use Surgery-GAN to generate large synthetic datasets which we then use to train five different segmentation models. Results demonstrate that using our synthetic images always improves the mean segmentation performance with respect to only using real images. For example, when considering radical prostatectomy, we can boost the mean segmentation performance by up to 5.43%. More interestingly, experimental results indicate that the performance improvement is larger in the set of classes that are under-represented in the training sets, where the performance boost of specific classes reaches up to 61.6%.

Anatomy Preserving GAN for Realistic Simulation of Intraoperative Liver Ultrasound Images.

OA-GAN: Organ-Aware Generative Adversarial Network for Synthesizing Contrast-Enhanced Medical Images

Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis

Towards Real-time Intrahepatic Vessel Identification in Intraoperative Ultrasound-Guided Liver Surgery

AwCPM-Net: A Collaborative Constraint GAN for 3D Coronary Artery Reconstruction in Intravascular Ultrasound Sequences.

Deep Attentional GAN-based High-Resolution Ultrasound Imaging

SA-GAN: Structure-Aware GAN for Organ-Preserving Synthetic CT Generation

Guided image generation for improved surgical image segmentation

Three-Dimensional Liver Image Segmentation Using Generative Adversarial Networks Based on Feature Restoration

Realistic Ultrasound Image Synthesis for Improved Classification of Liver Disease

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

From 2D Ultrasound to Patient-Specific 3D Surface Models for Interventional Guidance

Real-time ultrasound AR 3D visualization toward better topological structure perception for hepatobiliary surgery

Light&Fast Generative Adversarial Network for high-fidelity CT image synthesis of liver tumor

Paired conditional generative adversarial network for highly accelerated liver 4D MRI

FetalBrainAwareNet: Bridging GANs with anatomical insight for fetal ultrasound brain plane synthesis

Multi-phase Liver-Specific DCE-MRI Translation Via A Registration-Guided GAN

Points of interest linear attention network for real‐time non‐rigid liver volume to surface registration

Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks

Synthesis and Edition of Ultrasound Images via Sketch Guided Progressive Growing GANs

Anatomically Constrained Deformable 3D Reconstruction of Intraoperative Uterus from Preoperative MRI Data on Uterine Fibroid Treatment