Abstract:Background and Objective: Multi-modal medical images, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), have been widely used for the diagnosis of brain disorder diseases like Alzheimer's disease (AD) since they can provide various information. PET scans can detect cellular changes in organs and tissues earlier than MRI. Unlike MRI, PET data is difficult to acquire due to cost, radiation, or other limitations. Moreover, PET data is missing for many subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. To solve this problem, a 3D end-to-end generative adversarial network (named BPGAN) is proposed to synthesize brain PET from MRI scans, which can be used as a potential data completion scheme for multi-modal medical image research.Methods: We propose BPGAN, which learns an end-to-end mapping function to transform the input MRI scans to their underlying PET scans. First, we design a 3D multiple convolution U-Net (MCU) generator architecture to improve the visual quality of synthetic results while preserving the diverse brain structures of different subjects. By further employing a 3D gradient profile (GP) loss and structural similarity index measure (SSIM) loss, the synthetic PET scans have higher-similarity to the ground truth. In this study, we explore alternative data partitioning ways to study their impact on the performance of the proposed method in different medical scenarios.Results: We conduct experiments on a publicly available ADNI database. The proposed BPGAN is evaluated by mean absolute error (MAE), peak-signal-to-noise-ratio (PSNR) and SSIM, superior to other compared models in these quantitative evaluation metrics. Qualitative evaluations also validate the effectiveness of our approach. Additionally, combined with MRI and our synthetic PET scans, the accuracies of multi-class AD diagnosis on dataset-A and dataset-B are 85.00% and 56.47%, which have been improved by about 1% and 1%, respectively, compared to the stand-alone MRI. Conclusions: The experimental results of quantitative measures, qualitative displays, and classification evaluation demonstrate that the synthetic PET images by BPGAN are reasonable and high-quality, which provide complementary information to improve the performance of AD diagnosis. This work provides a valuable reference for multi-modal medical image analysis.(c) 2022 Elsevier B.V. All rights reserved.

Pyramid-attentive GAN for Multimodal Brain Image Complementation in Alzheimer’s Disease Classification

Task-Induced Pyramid and Attention GAN for Multimodal Brain Image Imputation and Classification in Alzheimer's Disease

Cross-modal Medical Image Generation Based on Pyramid Convolutional Attention Network

Multimodal transformer network for incomplete image generation and diagnosis of Alzheimer's disease

BPGAN: Brain PET synthesis from MRI using generative adversarial network for multi-modal Alzheimer's disease diagnosis

Multimodal 3D Convolutional Neural Networks for Classification of Brain Disease Using Structural MR and FDG-PET Images.

An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis

A coupled-GAN architecture to fuse MRI and PET image features for multi-stage classification of Alzheimer's disease

Synthesizing Missing PET from MRI with Cycle-consistent Generative Adversarial Networks for Alzheimer's Disease Diagnosis

Multi-modal data Alzheimer’s disease detection based on 3D convolution

Synthesizing PET Scans Using Attention GAN Based on MRI Scans and Alzheimer's Disease Information.

Multi-Modality Cascaded Convolutional Neural Networks for Alzheimer’s Disease Diagnosis

DMA-HPCNet: Dual Multi-Level Attention Hybrid Pyramid Convolution Neural Network for Alzheimer's Disease Classification

Multimodal Fusion Diagnosis of Alzheimer’s Disease Based on FDG-PET Generation

DMA-HPCNet: Dual Multi-level Attention Hybrid Pyramid Convolution Neural Network for Alzheimer’s Disease Classification#

Early diagnosis of Alzheimer's disease using dual GAN model with pyramid attention networks

Alzheimer's Disease Classification Accuracy is Improved by MRI Harmonization based on Attention-Guided Generative Adversarial Networks

MDMA: Multimodal Data and Multi-attention Based Deep Learning Model for Alzheimer’s Disease Diagnosis

Diagnosis of Alzheimer's Disease via Multi-modality 3D Convolutional Neural Network

Bidirectional Mapping of Brain MRI and PET with 3D Reversible GAN for the Diagnosis of Alzheimer's Disease.

Attention-based 3D CNN with Multi-layer Features for Alzheimer's Disease Diagnosis using Brain Images