Mask-conditioned latent diffusion for generating gastrointestinal polyp images

Roman Macháček,Leila Mozaffari,Zahra Sepasdar,Sravanthi Parasa,Pål Halvorsen,Michael A. Riegler,Vajira Thambawita

2023-04-11

Abstract:In order to take advantage of AI solutions in endoscopy diagnostics, we must overcome the issue of limited annotations. These limitations are caused by the high privacy concerns in the medical field and the requirement of getting aid from experts for the time-consuming and costly medical data annotation process. In computer vision, image synthesis has made a significant contribution in recent years as a result of the progress of generative adversarial networks (GANs) and diffusion probabilistic models (DPM). Novel DPMs have outperformed GANs in text, image, and video generation tasks. Therefore, this study proposes a conditional DPM framework to generate synthetic GI polyp images conditioned on given generated segmentation masks. Our experimental results show that our system can generate an unlimited number of high-fidelity synthetic polyp images with the corresponding ground truth masks of polyps. To test the usefulness of the generated data, we trained binary image segmentation models to study the effect of using synthetic data. Results show that the best micro-imagewise IOU of 0.7751 was achieved from DeepLabv3+ when the training data consists of both real data and synthetic data. However, the results reflect that achieving good segmentation performance with synthetic data heavily depends on model architectures.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the limited labeled data when using artificial intelligence technology in endoscopic diagnosis. Specifically, due to the high privacy requirements in the medical field and the time - consuming and costly process of obtaining medical data labeling from experts, the available labeled data is very limited. This restricts the performance of supervised learning models during training, especially in image generation and segmentation tasks. To overcome this challenge, the author proposes a framework based on the diffusion model for generating synthetic gastrointestinal polyp images conditioned on a given segmentation mask. Through this method, the research aims to generate an unlimited number of high - fidelity synthetic polyp images and their corresponding real - polyp masks, thereby enhancing the training data set and improving the performance of the model in polyp detection and segmentation tasks.

Mask-conditioned latent diffusion for generating gastrointestinal polyp images

Polyp-DDPM: Diffusion-Based Semantic Polyp Synthesis for Enhanced Segmentation

Using Diffusion Models to Generate Synthetic Labelled Data for Medical Image Segmentation

PolypConnect: Image inpainting for generating realistic gastrointestinal tract images with polyps

Examining the Effect of Synthetic Data Augmentation in Polyp Detection and Segmentation.

Probabilistic Modeling Ensemble Vision Transformer Improves Complex Polyp Segmentation

Generalize Polyp Segmentation via Inpainting across Diverse Backgrounds and Pseudo-Mask Refinement

Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention

MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generation

Real-Time Gastric Polyp Detection Using Convolutional Neural Networks

GAN-Based Synthetic Gastrointestinal Image Generation

Highlighted Diffusion Model as Plug-in Priors for Polyp Segmentation

Using diffusion models to generate synthetic labeled data for medical image segmentation

Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models

Less is More: Unsupervised Mask-guided Annotated CT Image Synthesis with Minimum Manual Segmentations

Mask-guided cross-image attention for zero-shot in-silico histopathologic image generation with a diffusion model

Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy

DEPAS: De-novo Pathology Semantic Masks using a Generative Model

Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology

Synthesis of Annotated Colorectal Cancer Tissue Images from Gland Layout

Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation