Abstract:This paper presents the second-placed solution for task 8 and the participation solution for task 7 of BraTS 2024. The adoption of automated brain analysis algorithms to support clinical practice is increasing. However, many of these algorithms struggle with the presence of brain lesions or the absence of certain MRI modalities. The alterations in the brain's morphology leads to high variability and thus poor performance of predictive models that were trained only on healthy brains. The lack of information that is usually provided by some of the missing MRI modalities also reduces the reliability of the prediction models trained with all modalities. In order to improve the performance of these models, we propose the use of conditional 3D wavelet diffusion models. The wavelet transform enabled full-resolution image training and prediction on a GPU with 48 GB VRAM, without patching or downsampling, preserving all information for prediction. For the inpainting task of BraTS 2024, the use of a large and variable number of healthy masks and the stability and efficiency of the 3D wavelet diffusion model resulted in 0.007, 22.61 and 0.842 in the validation set and 0.07 , 22.8 and 0.91 in the testing set (MSE, PSNR and SSIM respectively). The code for these tasks is available at <a class="link-external link-https" href="https://github.com/ShadowTwin41/BraTS_2023_2024_solutions" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

This paper attempts to solve two main problems: 1. **Brain Tumor Resection**: In MRI images, how to effectively remove the brain tumor area and generate healthy brain tissue to replace these areas. This task is called "Local Synthesis", that is, to fill the removed tumor area through inpainting technology to restore the image to a healthy state. 2. **Missing - modality Generation**: In multi - modal MRI data, how to generate the missing - modality images when some modalities are missing. This task is called "Global Synthesis", that is, to generate the missing fourth modality (such as T1 - weighted enhanced image) according to the other three existing modalities (such as T1, T2 and FLAIR). ### Background and Challenges - **Brain Tumor Resection**: Existing automatic brain analysis algorithms perform poorly when dealing with images containing brain lesions because the morphological changes caused by the lesions increase the complexity and instability of the prediction model. In addition, if some MRI modalities are missing, it will also reduce the reliability of the model. - **Missing - modality Generation**: In clinical practice, obtaining complete four - modal MRI data is very time - consuming and costly. Some patients are difficult to obtain high - quality MRI data due to claustrophobia, mental disorders or excessive movement during the scanning process. For patients with metal implants, it may even be impossible to perform MRI scans. Therefore, how to generate the missing modality when some modalities are missing has become an important research direction. ### Solutions To meet the above challenges, the author proposes a method based on the Conditional 3D Wavelet Diffusion Model (3D WDM). Specifically: - **3D Wavelet Diffusion Model**: Use wavelet transform to reduce the spatial dimension of the image by 8 times, so that it can process the image at full resolution on a GPU with 48 GB of video memory without blocking or downsampling. - **Conditional Sampling**: During the training and inference process, the model uses a variety of different healthy regions as input and adopts different schedulers and sampling pipelines to improve the quality and efficiency of the generated images. ### Experimental Results - **Local Synthesis Task**: The results on the validation set show that the best model (AKH l5000) reaches 0.07 ± 0.04, 22.8 ± 4.41 and 0.91 ± 0.15 in terms of MSE, PSNR and SSIM respectively. - **Global Synthesis Task**: Through visual inspection, the model can generate realistic missing - modality images. Increasing the sampling steps can further improve the quality of the generated images, but it will also increase the computing time. ### Conclusion This paper effectively solves the problems of brain tumor resection and missing - modality generation by proposing a method based on the 3D wavelet diffusion model. The experimental results show that this method performs well in terms of image quality and generation efficiency, providing a new solution for clinical applications.

Brain Tumour Removing and Missing Modality Generation using 3D WDM

Improved Multi-Task Brain Tumour Segmentation with Synthetic Data Augmentation

How we won BraTS 2023 Adult Glioma challenge? Just faking it! Enhanced Synthetic Data Augmentation and Model Ensemble for brain tumour segmentation

M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities

Optimizing Brain Tumor Segmentation with MedNeXt: BraTS 2024 SSA and Pediatrics

The Brain Tumor Segmentation (BraTS) Challenge: Local Synthesis of Healthy Brain Tissue via Inpainting

Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge

Multimodal CNN Networks for Brain Tumor Segmentation in MRI: A BraTS 2022 Challenge Solution

The 2024 Brain Tumor Segmentation (BraTS) Challenge: Glioma Segmentation on Post-treatment MRI

The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification

The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI

Redundancy Reduction in Semantic Segmentation of 3D Brain Tumor MRIs

cWDM: Conditional Wavelet Diffusion Models for Cross-Modality 3D Medical Image Synthesis

Attention-Enhanced Hybrid Feature Aggregation Network for 3D Brain Tumor Segmentation

The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients

The Brain Tumor Segmentation (BraTS) Challenge 2023: Brain MR Image Synthesis for Tumor Segmentation (BraSyn)

Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge

The ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: Intracranial Meningioma

Generalized Wasserstein Dice Loss, Test-time Augmentation, and Transformers for the BraTS 2021 challenge

Reciprocal Adversarial Learning for Brain Tumor Segmentation: A Solution to BraTS Challenge 2021 Segmentation Task

Synthesizing Missing MRI Sequences from Available Modalities using Generative Adversarial Networks in BraTS Dataset