Abstract:The lack of large datasets and high-quality annotated data often limits the development of accurate and robust machine-learning models within the medical and surgical domains. In the machine learning community, generative models have recently demonstrated that it is possible to produce novel and diverse synthetic images that closely resemble reality while controlling their content with various types of annotations. However, generative models have not been yet fully explored in the surgical domain, partially due to the lack of large datasets and due to specific challenges present in the surgical domain such as the large anatomical diversity. We propose Surgery-GAN, a novel generative model that produces synthetic images from segmentation maps. Our architecture produces surgical images with improved quality when compared to early generative models thanks to the combination of channel- and pixel-level normalization layers that boost image quality while granting adherence to the input segmentation map. While state-of-the-art generative models often generate overfitted images, lacking diversity, or containing unrealistic artefacts such as cartooning; experiments demonstrate that Surgery-GAN is able to generate novel, realistic, and diverse surgical images in three different surgical datasets: cholecystectomy, partial nephrectomy, and radical prostatectomy. In addition, we investigate whether the use of synthetic images together with real ones can be used to improve the performance of other machine-learning models. Specifically, we use Surgery-GAN to generate large synthetic datasets which we then use to train five different segmentation models. Results demonstrate that using our synthetic images always improves the mean segmentation performance with respect to only using real images. For example, when considering radical prostatectomy, we can boost the mean segmentation performance by up to 5.43%. More interestingly, experimental results indicate that the performance improvement is larger in the set of classes that are under-represented in the training sets, where the performance boost of specific classes reaches up to 61.6%.

Generating Large Labeled Data Sets for Laparoscopic Image Processing Tasks Using Unpaired Image-to-Image Translation

Navigating the Synthetic Realm: Harnessing Diffusion-based Models for Laparoscopic Text-to-Image Generation

SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Models

SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models

Guided image generation for improved surgical image segmentation

Exploring semantic consistency in unpaired image translation to generate data for surgical applications

Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention

SSIS-Seg: Simulation-Supervised Image Synthesis for Surgical Instrument Segmentation

Reducing annotating load: Active learning with synthetic images in surgical instrument segmentation

Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models

Realistic Surgical Image Dataset Generation Based On 3D Gaussian Splatting

Heidelberg colorectal data set for surgical data science in the sensor operating room

LapSeg3D: Weakly Supervised Semantic Segmentation of Point Clouds Representing Laparoscopic Scenes

Strategies to Improve Real-World Applicability of Laparoscopic Anatomy Segmentation Models

Unpaired Translation from Semantic Label Maps to Images by Leveraging Domain-Specific Simulations

Utilizing Synthetic Data for Medical Vision-Language Pre-training: Bypassing the Need for Real Images

SurgAI3.8K: A Labeled Dataset of Gynecologic Organs in Laparoscopy with Application to Automatic Augmented Reality Surgical Guidance

LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images

Interactive Generation of Laparoscopic Videos with Diffusion Models

Using diffusion models to generate synthetic labeled data for medical image segmentation

Using Diffusion Models to Generate Synthetic Labelled Data for Medical Image Segmentation