Abstract:Background and Objective: Multi-modal medical images, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), have been widely used for the diagnosis of brain disorder diseases like Alzheimer's disease (AD) since they can provide various information. PET scans can detect cellular changes in organs and tissues earlier than MRI. Unlike MRI, PET data is difficult to acquire due to cost, radiation, or other limitations. Moreover, PET data is missing for many subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. To solve this problem, a 3D end-to-end generative adversarial network (named BPGAN) is proposed to synthesize brain PET from MRI scans, which can be used as a potential data completion scheme for multi-modal medical image research.Methods: We propose BPGAN, which learns an end-to-end mapping function to transform the input MRI scans to their underlying PET scans. First, we design a 3D multiple convolution U-Net (MCU) generator architecture to improve the visual quality of synthetic results while preserving the diverse brain structures of different subjects. By further employing a 3D gradient profile (GP) loss and structural similarity index measure (SSIM) loss, the synthetic PET scans have higher-similarity to the ground truth. In this study, we explore alternative data partitioning ways to study their impact on the performance of the proposed method in different medical scenarios.Results: We conduct experiments on a publicly available ADNI database. The proposed BPGAN is evaluated by mean absolute error (MAE), peak-signal-to-noise-ratio (PSNR) and SSIM, superior to other compared models in these quantitative evaluation metrics. Qualitative evaluations also validate the effectiveness of our approach. Additionally, combined with MRI and our synthetic PET scans, the accuracies of multi-class AD diagnosis on dataset-A and dataset-B are 85.00% and 56.47%, which have been improved by about 1% and 1%, respectively, compared to the stand-alone MRI. Conclusions: The experimental results of quantitative measures, qualitative displays, and classification evaluation demonstrate that the synthetic PET images by BPGAN are reasonable and high-quality, which provide complementary information to improve the performance of AD diagnosis. This work provides a valuable reference for multi-modal medical image analysis.(c) 2022 Elsevier B.V. All rights reserved.

Image Augmentation Using a Task Guided Generative Adversarial Network for Age Estimation on Brain MRI

Multi-modal Perceptual Adversarial Learning for Longitudinal Prediction of Infant MR Images

Enhancing magnetic resonance imaging-driven Alzheimer’s disease classification performance using generative adversarial learning

A novel locally-constrained GAN-based ensemble to synthesize arterial spin labeling images

BPGAN: Brain PET synthesis from MRI using generative adversarial network for multi-modal Alzheimer's disease diagnosis

Enhancing Brain Age Estimation with a Multimodal 3D CNN Approach Combining Structural MRI and AI-Synthesized Cerebral Blood Volume Data

Prediction of Alzheimer's Disease Progression with Multi-Information Generative Adversarial Network

MRI Volume-Based Robust Brain Age Estimation Using Weight-Shared Spatial Attention in 3D CNNs

Brain Status Transferring Generative Adversarial Network for Decoding Individualized Atrophy in Alzheimer's Disease

Graph Transformer Geometric Learning of Brain Networks Using Multimodal MR Images for Brain Age Estimation

Task-Induced Pyramid and Attention GAN for Multimodal Brain Image Imputation and Classification in Alzheimer's Disease

Bayesian Conditional GAN for MRI Brain Image Synthesis

Cross-Modality Augmentation Of Brain Mr Images Using A Novel Pairwise Generative Adversarial Network For Enhanced Glioma Classification

Augmenting healthy brain magnetic resonance images using generative adversarial networks

Brain Age Estimation From MRI Using Cascade Networks with Ranking Loss

MRI to PET Cross-Modality Translation using Globally and Locally Aware GAN (GLA-GAN) for Multi-Modal Diagnosis of Alzheimer's Disease

Longitudinal Prediction of Infant MR Images With Multi-Contrast Perceptual Adversarial Learning

Dual Graph Attention based Disentanglement Multiple Instance Learning for Brain Age Estimation

Age Estimation from Brain MRI Images Using Deep Learning

Learning to Synthesize 7 T MRI from 3 T MRI with Few Data by Deformable Augmentation.

Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks