Slice-Consistent 3D Volumetric Brain CT-to-MRI Translation with 2D Brownian Bridge Diffusion Model

Kyobin Choo,Youngjun Jun,Mijin Yun,Seong Jae Hwang

2024-07-06

Abstract:In neuroimaging, generally, brain CT is more cost-effective and accessible imaging option compared to MRI. Nevertheless, CT exhibits inferior soft-tissue contrast and higher noise levels, yielding less precise structural clarity. In response, leveraging more readily available CT to construct its counterpart MRI, namely, medical image-to-image translation (I2I), serves as a promising solution. Particularly, while diffusion models (DMs) have recently risen as a powerhouse, they also come with a few practical caveats for medical I2I. First, DMs' inherent stochasticity from random noise sampling cannot guarantee consistent MRI generation that faithfully reflects its CT. Second, for 3D volumetric images which are prevalent in medical imaging, naively using 2D DMs leads to slice inconsistency, e.g., abnormal structural and brightness changes. While 3D DMs do exist, significant training costs and data dependency bring hesitation. As a solution, we propose novel style key conditioning (SKC) and inter-slice trajectory alignment (ISTA) sampling for the 2D Brownian bridge diffusion model. Specifically, SKC ensures a consistent imaging style (e.g., contrast) across slices, and ISTA interconnects the independent sampling of each slice, deterministically achieving style and shape consistent 3D CT-to-MRI translation. To the best of our knowledge, this study is the first to achieve high-quality 3D medical I2I based only on a 2D DM with no extra architectural models. Our experimental results show superior 3D medical I2I than existing 2D and 3D baselines, using in-house CT-MRI dataset and BraTS2023 FLAIR-T1 MRI dataset.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the issue of synthesizing Magnetic Resonance Imaging (MRI) from Computed Tomography (CT) images in medical imaging, particularly in brain imaging. Although CT is commonly used in neuroimaging due to its cost-effectiveness and accessibility, it performs poorly in terms of soft tissue contrast and noise level, resulting in lower structural clarity. To overcome this limitation, researchers have explored using CT images to generate corresponding MRI images, a process known as "image-to-image translation" (I2I). Specifically, the paper proposes a method based on the 2D Brownian Bridge Diffusion Model (BBDM), which introduces Style Key Conditioning (SKC) and Inter-Slice Trajectory Alignment (ISTA) techniques to achieve high-quality and consistent 3D brain CT to MRI conversion.

The main contributions of the paper include:

1. The introduction of SKC and ISTA methods, using a 2D diffusion model to achieve 3D volumetric brain CT to MRI conversion, addressing the issues of MRI generation inconsistency caused by random noise sampling and slice inconsistency in 3D volumetric images.

2. In combination with BBDM, a fully deterministic and reliable medical image-to-image translation is achieved, where the structure and style of the target volume are controlled by the source volume and histogram conditioning, respectively.

3. Experimental results on an internal CT-MRI dataset and the public BraTS2023 FLAIR-T1 MRI dataset show that the proposed 2D method outperforms existing 2D and 3D baseline models in cross-modal image translation tasks, without the need for additional architectural models such as autoencoders.

Through these innovations, the paper not only improves the quality of 3D medical image-to-image translation but also enhances the reliability and efficiency of medical applications.