3D MRI Synthesis with Slice-Based Latent Diffusion Models: Improving Tumor Segmentation Tasks in Data-Scarce Regimes

Aghiles Kebaili,Jérôme Lapuyade-Lahorgue,Pierre Vera,Su Ruan

2024-06-08

Abstract:Despite the increasing use of deep learning in medical image segmentation, the limited availability of annotated training data remains a major challenge due to the time-consuming data acquisition and privacy regulations. In the context of segmentation tasks, providing both medical images and their corresponding target masks is essential. However, conventional data augmentation approaches mainly focus on image synthesis. In this study, we propose a novel slice-based latent diffusion architecture designed to address the complexities of volumetric data generation in a slice-by-slice fashion. This approach extends the joint distribution modeling of medical images and their associated masks, allowing a simultaneous generation of both under data-scarce regimes. Our approach mitigates the computational complexity and memory expensiveness typically associated with diffusion models. Furthermore, our architecture can be conditioned by tumor characteristics, including size, shape, and relative position, thereby providing a diverse range of tumor variations. Experiments on a segmentation task using the BRATS2022 confirm the effectiveness of the synthesized volumes and masks for data augmentation.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily aims to address the issue of data scarcity in medical image segmentation tasks. Specifically, the research team proposes a Slice-Based Latent Diffusion Model (SBLDM) designed to generate high-quality 3D MRI images and their corresponding tumor segmentation masks. This method is particularly suitable for situations with limited data. Its main contributions include: 1. **Efficient Generation Mechanism**: Compared to traditional pixel-space diffusion models or standard latent diffusion models, SBLDM significantly reduces the computational and memory resource requirements when generating 3D MRI images and their segmentation masks. 2. **Data Augmentation Effect**: By synthesizing data to enhance the training set, the performance of the segmentation task is improved. Experimental results show that the model enhanced with data augmentation outperforms other methods, especially in terms of Dice Similarity Coefficient (DSC) and Intersection over Union (IoU). 3. **Conditional Control Capability**: The model allows users to perform conditional generation based on tumor characteristics (such as size, shape, and relative position), thereby producing diverse tumor variants. This conditional setting also acts as a regularization mechanism. In summary, the paper aims to overcome the issue of data scarcity in the field of medical image segmentation through a novel approach, while ensuring the quality and diversity of the generated images, thereby enhancing the model's generalization ability and segmentation accuracy.

3D MRI Synthesis with Slice-Based Latent Diffusion Models: Improving Tumor Segmentation Tasks in Data-Scarce Regimes

Synthesizing 3D Multi-Contrast Brain Tumor MRIs Using Tumor Mask Conditioning

Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis

Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis

Synthetic MRI in action: A novel framework in data augmentation strategies for robust multi-modal brain tumor segmentation

Improvement of Multiparametric MR Image Segmentation by Augmenting the Data With Generative Adversarial Networks for Glioma Patients

Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation

Enhanced segmentation of femoral bone metastasis in CT scans of patients using synthetic data generation with 3D diffusion models

Med-DANet: Dynamic Architecture Network for Efficient Medical Volumetric Segmentation

A 3D generative model of pathological multi-modal MR images and segmentations

Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models

Denoising diffusion probabilistic models for 3D medical image generation

Mask-guided Data Augmentation for Multiparametric MRI Generation with a Rare Hepatocellular Carcinoma

Generative Enhancement for 3D Medical Images

X-Diffusion: Generating Detailed 3D MRI Volumes From a Single Image Using Cross-Sectional Diffusion Models

Discriminative Hamiltonian Variational Autoencoder for Accurate Tumor Segmentation in Data-Scarce Regimes

Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models

MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generation

An Augmentation Strategy for Medical Image Processing Based on Statistical Shape Model and 3D Thin Plate Spline for Deep Learning

Advanced image generation for cancer using diffusion models

Generating multi-pathological and multi-modal images and labels for brain MRI