Abstract:The infant brain undergoes a remarkable period of neural development that is crucial for the development of cognitive and behavioral capacities (Hasegawa et al., 2018 ). Longitudinal magnetic resonance imaging (MRI) is able to characterize the developmental trajectories and is critical in neuroimaging studies of early brain development. However, missing data at different time points is an unavoidable occurrence in longitudinal studies owing to participant attrition and scan failure. Compared to dropping incomplete data, data imputation is considered a better solution to address such missing data in order to preserve all available samples. In this paper, we adapt generative adversarial networks (GAN) to a new application: longitudinal image prediction of structural MRI in the first year of life. In contrast to existing medical image-to-image translation applications of GANs, where inputs and outputs share a very close anatomical structure, our task is more challenging as brain size, shape and tissue contrast vary significantly between the input data and the predicted data. Several improvements over existing GAN approaches are proposed to address these challenges in our task. To enhance the realism, crispness, and accuracy of the predicted images, we incorporate both a traditional voxel-wise reconstruction loss as well as a perceptual loss term into the adversarial learning scheme. As the differing contrast changes in T1w and T2w MR images in the first year of life, we incorporate multi-contrast images leading to our proposed 3D multi-contrast perceptual adversarial network (MPGAN). Extensive evaluations are performed to assess the qualityand fidelity of the predicted images, including qualitative and quantitative assessments of the image appearance, as well as quantitative assessment on two segmentation tasks. Our experimental results show that our MPGAN is an effective solution for longitudinal MR image data imputation in the infant brain. We further apply our predicted/imputed images to two practical tasks, a regression task and a classification task, in order to highlight the enhanced task-related performance following image imputation. The results show that the model performance in both tasks is improved by including the additional imputed data, demonstrating the usability of the predicted images generated from our approach.

Multi-modal Perceptual Adversarial Learning for Longitudinal Prediction of Infant MR Images

Longitudinal Prediction of Infant MR Images With Multi-Contrast Perceptual Adversarial Learning

Longitudinal Prediction of Postnatal Brain Magnetic Resonance Images via a Metamorphic Generative Adversarial Network

Longitudinal Prediction of Infant Diffusion MRI Data Via Graph Convolutional Adversarial Networks

LDGAN: Longitudinal-Diagnostic Generative Adversarial Network for Disease Progression Prediction with Missing Structural MRI

Development-driven Diffusion Model for Longitudinal Prediction of Fetal Brain MRI with Unpaired Data

Longitudinal Structural MRI Data Prediction in Nondemented and Demented Older Adults Via Generative Adversarial Convolutional Network

Prediction of Alzheimer's Disease Progression with Multi-Information Generative Adversarial Network

Graph-Based Deep Learning for Prediction of Longitudinal Infant Diffusion MRI Data

Joint Prediction of Longitudinal Development of Cortical Surfaces and White Matter Fibers from Neonatal MRI

Image Augmentation Using a Task Guided Generative Adversarial Network for Age Estimation on Brain MRI

Predictive Image Regression for Longitudinal Studies with Missing Data

Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment

TR-GAN: Multi-session Future MRI Prediction with Temporal Recurrent Generative Adversarial Network

Deep Multimodal Learning From MRI and Clinical Data for Early Prediction of Neurodevelopmental Deficits in Very Preterm Infants

Deep Generative Adversarial Networks for Thin-Section Infant MR Image Reconstruction

Adversarial Learning for MRI Reconstruction and Classification of Cognitively Impaired Individuals

TR-TransGAN: Temporal Recurrent Transformer Generative Adversarial Network for Longitudinal MRI Dataset Expansion

DualMMP-GAN: Dual-scale Multi-Modality Perceptual Generative Adversarial Network for Medical Image Segmentation

Temporally Adjustable Longitudinal Fluid-Attenuated Inversion Recovery MRI Estimation / Synthesis for Multiple Sclerosis

Transformer based multi-modal MRI fusion for prediction of post-menstrual age and neonatal brain development analysis