Abstract:Contrast-enhanced brain MRI (CE-MRI) is a valuable diagnostic technique but may pose health risks and incur high costs. To create safer alternatives, multi-modality medical image translation aims to synthesize CE-MRI images from other available modalities. Although existing methods can generate promising predictions, they still face two challenges, i.e., exhibiting over-confidence and lacking interpretability on predictions. To address the above challenges, this paper introduces TrustI2I, a novel trustworthy method that reformulates multi-to-one medical image translation problem as a multimodal regression problem, aiming to build an uncertainty-aware and reliable system. Specifically, our method leverages deep evidential regression to estimate prediction uncertainties and employs an explicit intermediate and late fusion strategy based on the Mixture of Normal Inverse Gamma (MoNIG) distribution, enhancing both synthesis quality and interpretability. Additionally, we incorporate uncertainty calibration to improve the reliability of uncertainty. Validation on the BraTS2018 dataset demonstrates that our approach surpasses current methods, producing higher-quality images with rational uncertainty estimation.

What problem does this paper attempt to address?

The paper aims to address two major issues in contrast-enhanced brain magnetic resonance imaging (CE-MRI) synthesis: insufficient uncertainty estimation and poor interpretability of the prediction results. The proposed method targets contrast-enhanced brain magnetic resonance imaging (CE-MRI), a highly valuable diagnostic technique that poses health risks and high costs in practical applications. To address these issues, researchers have proposed multimodal medical image translation techniques to synthesize CE-MRI images from other available modalities, providing a safer and more cost-effective alternative. Existing multimodal medical image translation methods, while capable of generating satisfactory prediction results, still face two major challenges: 1. **Insufficient Uncertainty Estimation**: Existing methods struggle to identify situations where reliable predictions cannot be made. That is, these methods find it difficult to answer questions like "Can we trust the generated results?" and "Are these results reliable enough for diagnosis?" 2. **Lack of Interpretability**: The implicit nature of the modality fusion process undermines its interpretability, meaning the details of how different modalities are integrated and how they collectively contribute to the final prediction are unclear. To address these challenges, the paper proposes a new method called TrustI2I, which redefines the multimodal-to-single-modal medical image translation problem as a multimodal regression problem, aiming to build a system that can perceive uncertainty and be reliable. Specifically, the method utilizes deep evidence regression to estimate prediction uncertainty and employs explicit intermediate fusion and late fusion strategies based on the Normal Inverse Gamma (MoNIG) distribution to improve the quality and interpretability of the synthesized images. Additionally, an uncertainty calibration strategy is introduced to enhance the reliability of uncertainty estimation. In summary, the main goal of the paper is to improve the uncertainty and interpretability of multimodal medical image translation by proposing a new, trustworthy method, thereby enhancing the quality and reliability of the synthesized images.

Trustworthy Contrast-enhanced Brain MRI Synthesis

A Novel Method Of Synthetic Ct Generation From Mr Images Based On Convolutional Neural Networks

Synthesizing Contrast-enhanced Computed Tomography Images with an Improved Conditional Generative Adversarial Network

Uncertainty Estimation in Contrast-Enhanced MR Image Translation with Multi-Axis Fusion

Deep learning-based 3D MRI contrast-enhanced synthesis from a 2D noncontrast T2Flair sequence

A Unified Conditional Disentanglement Framework for Multimodal Brain MR Image Translation

Cross-Modality Image Translation: CT Image Synthesis of MR Brain Images using Multi Generative Network with Perceptual Supervision

Multi-Modality MR Image Synthesis via Confidence-Guided Aggregation and Cross-Modality Refinement

Deep Learning‐Based 3D MRI Contrast‐Enhanced Synthesis From A 2D Non‐contrast T2Flair Sequence Deep Learning, MR images Synthesis

Synthesis of Contrast-Enhanced Breast MRI Using Multi-b-Value DWI-based Hierarchical Fusion Network with Attention Mechanism

Generalized Brain Image Synthesis with Transferable Convolutional Sparse Coding Networks.

MR-Contrast-Aware Image-to-Image Translations with Generative Adversarial Networks

Synthesize High-Quality Multi-Contrast Magnetic Resonance Imaging from Multi-Echo Acquisition Using Multi-Task Deep Generative Model

Towards General Text-guided Image Synthesis for Customized Multimodal Brain MRI Generation

Disentangled Multimodal Brain MR Image Translation via Transformer-based Modality Infuser

Local Implicit Neural Representations for Multi-Sequence MRI Translation

DS3-Net: Difficulty-perceived Common-to-T1ce Semi-Supervised Multimodal MRI Synthesis Network

UltimateSynth: MRI Physics for Pan-Contrast AI

Bidirectional feature matching based on deep pairwise contrastive learning for multiparametric MRI image synthesis

Generalizable synthetic MRI with physics-informed convolutional networks

Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models