Leveraging Multimodal CycleGAN for the Generation of Anatomically Accurate Synthetic CT Scans from MRIs

Leonardo Crespi,Samuele Camnasio,Damiano Dei,Nicola Lambri,Pietro Mancosu,Marta Scorsetti,Daniele Loiacono

2024-07-16

Abstract:In many clinical settings, the use of both Computed Tomography (CT) and Magnetic Resonance (MRI) is necessary to pursue a thorough understanding of the patient's anatomy and to plan a suitable therapeutical strategy; this is often the case in MRI-based radiotherapy, where CT is always necessary to prepare the dose delivery, as it provides the essential information about the radiation absorption properties of the tissues. Sometimes, MRI is preferred to contour the target volumes. However, this approach is often not the most efficient, as it is more expensive, time-consuming and, most importantly, stressful for the patients. To overcome this issue, in this work, we analyse the capabilities of different configurations of Deep Learning models to generate synthetic CT scans from MRI, leveraging the power of Generative Adversarial Networks (GANs) and, in particular, the CycleGAN architecture, capable of working in an unsupervised manner and without paired images, which were not available. Several CycleGAN models were trained unsupervised to generate CT scans from different MRI modalities with and without contrast agents. To overcome the problem of not having a ground truth, distribution-based metrics were used to assess the model's performance quantitatively, together with a qualitative evaluation where physicians were asked to differentiate between real and synthetic images to understand how realistic the generated images were. The results show how, depending on the input modalities, the models can have very different performances; however, models with the best quantitative results, according to the distribution-based metrics used, can generate very difficult images to distinguish from the real ones, even for physicians, demonstrating the approach's potential.

Image and Video Processing,Artificial Intelligence,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the following main issues:

### Core Issues

- **Reducing Patient Burden**: Alleviating the physiological and psychological stress on patients by reducing the number of medical imaging examinations they need to undergo, especially for those already in a vulnerable health state (such as hematology patients undergoing bone marrow transplant treatment).
- **Improving Efficiency and Reducing Costs**: Lowering medical costs and shortening treatment preparation time by reducing the need for two different medical imaging technologies (CT and MRI).
- **Improving Medical Processes**: Avoiding the additional time and economic costs associated with performing two imaging sessions and reducing potential errors in the image registration process.

### Technical Approaches

- **Utilizing Deep Learning**: Researching the use of Generative Adversarial Networks (GANs), particularly the CycleGAN architecture, to automatically generate synthetic CT images from MRI images. This technology can work without paired data, meaning it does not require corresponding CT and MRI images of the same patient at the same time point.
- **Multimodal Image Processing**: Exploring how to combine different MRI modalities (such as T1-weighted, T2-weighted, etc.) to generate more accurate synthetic CT images.

### Objectives

- **Generating High-Quality Synthetic CT Images**: Ensuring that the generated CT images are as anatomically accurate as possible, including the geometric shapes, densities, and texture features of tissues.
- **Evaluating Model Performance**: Using quantitative methods (such as distribution similarity metrics) and qualitative methods (physician assessments) to evaluate the quality and authenticity of the generated images.
- **Application Scope**: Focusing on the abdominal area, including key organs such as the liver, spleen, kidneys, and lower lungs, as the imaging needs of these areas have broad clinical significance.

By addressing the above issues, this research aims to provide a more efficient and patient-friendly solution for applications such as radiotherapy, while also introducing new data augmentation techniques and synthetic image generation methods to the field of medical imaging.