Abstract:Objective.Head and neck radiotherapy planning requires electron densities from different tissues for dose calculation. Dose calculation from imaging modalities such as MRI remains an unsolved problem since this imaging modality does not provide information about the density of electrons.Approach.We propose a generative adversarial network (GAN) approach that synthesizes CT (sCT) images from T1-weighted MRI acquisitions in head and neck cancer patients. Our contribution is to exploit new features that are relevant for improving multimodal image synthesis, and thus improving the quality of the generated CT images. More precisely, we propose a Dual branch generator based on the U-Net architecture and on an augmented multi-planar branch. The augmented branch learns specific 3D dynamic features, which describe the dynamic image shape variations and are extracted from different view-points of the volumetric input MRI. The architecture of the proposed model relies on an end-to-end convolutional U-Net embedding network.Results.The proposed model achieves a mean absolute error (MAE) of18.76±5.167in the target Hounsfield unit (HU) space on sagittal head and neck patients, with a mean structural similarity (MSSIM) of0.95±0.09and a Frechet inception distance (FID) of145.60±8.38. The model yields a MAE of26.83±8.27to generate specific primary tumor regions on axial patient acquisitions, with a Dice score of0.73±0.06and a FID distance equal to122.58±7.55. The improvement of our model over other state-of-the-art GAN approaches is of 3.8%, on a tumor test set. On both sagittal and axial acquisitions, the model yields the best peak signal-to-noise ratio of27.89±2.22and26.08±2.95to synthesize MRI from CT input.Significance.The proposed model synthesizes both sagittal and axial CT tumor images, used for radiotherapy treatment planning in head and neck cancer cases. The performance analysis across different imaging metrics and under different evaluation strategies demonstrates the effectiveness of our dual CT synthesis model to produce high quality sCT images compared to other state-of-the-art approaches. Our model could improve clinical tumor analysis, in which a further clinical validation remains to be explored.

Multimodal image synthesis based on disentanglement representations of anatomical and modality specific features, learned using uncooperative relativistic GAN

A Novel Method Of Synthetic Ct Generation From Mr Images Based On Convolutional Neural Networks

Multimodal MR Image Synthesis Using Gradient Prior and Adversarial Learning

Multi-Modal MRI Image Synthesis via GAN With Multi-Scale Gate Mergence

LR-cGAN: Latent representation based conditional generative adversarial network for multi-modality MRI synthesis

Disentangled Multimodal Brain MR Image Translation via Transformer-based Modality Infuser

A least square generative network based on invariant contrastive feature pair learning for multimodal MR image synthesis

A Unified Conditional Disentanglement Framework for Multimodal Brain MR Image Translation

Leveraging Multimodal CycleGAN for the Generation of Anatomically Accurate Synthetic CT Scans from MRIs

Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis

Multi-planar dual adversarial network based on dynamic 3D features for MRI-CT head and neck image synthesis

Disentangle, align and fuse for multimodal and semi-supervised image segmentation

SA-GAN: Structure-Aware GAN for Organ-Preserving Synthetic CT Generation

Attention-Guided Generative Adversarial Network to Address Atypical Anatomy in Modality Transfer

Paired-unpaired Unsupervised Attention Guided GAN with transfer learning for bidirectional brain MR-CT synthesis

Unified Multi-Modal Image Synthesis for Missing Modality Imputation

AutoEncoder-Driven Multimodal Collaborative Learning for Medical Image Synthesis

Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis

Mask-guided generative adversarial network for MRI-based CT synthesis

TCGAN: a transformer-enhanced GAN for PET synthetic CT

A Deep Learning Model for Multi-Domain MRI Synthesis Using Generative Adversarial Networks