CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis

Lan Jiang,Ye Mao,Xi Chen,Xiangfeng Wang,Chao Li

2023-03-24

Abstract:MRI synthesis promises to mitigate the challenge of missing MRI modality in clinical practice. Diffusion model has emerged as an effective technique for image synthesis by modelling complex and variable data distributions. However, most diffusion-based MRI synthesis models are using a single modality. As they operate in the original image domain, they are memory-intensive and less feasible for multi-modal synthesis. Moreover, they often fail to preserve the anatomical structure in MRI. Further, balancing the multiple conditions from multi-modal MRI inputs is crucial for multi-modal synthesis. Here, we propose the first diffusion-based multi-modality MRI synthesis model, namely Conditioned Latent Diffusion Model (CoLa-Diff). To reduce memory consumption, we design CoLa-Diff to operate in the latent space. We propose a novel network architecture, e.g., similar cooperative filtering, to solve the possible compression and noise in latent space. To better maintain the anatomical structure, brain region masks are introduced as the priors of density distributions to guide diffusion process. We further present auto-weight adaptation to employ multi-modal information effectively. Our experiments demonstrate that CoLa-Diff outperforms other state-of-the-art MRI synthesis methods, promising to serve as an effective tool for multi-modal MRI synthesis.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of multi - modal Magnetic Resonance Imaging (Multi - Modal MRI) synthesis in clinical practice. Specifically, when certain MRI modalities are missing, how to use the existing modality data to generate the missing modality images. The paper points out that the existing MRI synthesis methods based on the Diffusion Model (DM) mainly focus on single - modal synthesis. These methods operate in the original image domain, resulting in large memory consumption and difficulty in handling multi - modal synthesis. In addition, these methods often fail to well maintain the integrity of anatomical structures and are difficult to balance multiple conditions when dealing with multi - modal input conditions.

To overcome these problems, the authors propose a new method named Conditioned Latent Diffusion Model (CoLa - Diff). The main features of this model include:

1. **Latent - space operation**: By performing diffusion operations in the latent space, memory consumption is reduced, making multi - modal synthesis more feasible.
2. **Structure guidance**: The brain region mask is introduced as a prior to guide the diffusion process in order to better maintain the integrity of anatomical structures.
3. **Automatic weight adjustment**: An automatic weight adjustment mechanism is proposed to effectively utilize multi - modal information and balance multiple conditions.

The experimental results show that CoLa - Diff performs excellently in the multi - modal MRI synthesis task and is superior to other state - of - the - art methods. This makes CoLa - Diff expected to become an effective tool for generating MRI images, reducing the burden of MRI scans, and thus benefiting patients and medical providers.