Abstract:Mounting evidence has demonstrated that complex brain function processes are realized by the interaction of holistic functional brain networks which are spatially distributed across specific brain regions in a temporally dynamic fashion. Therefore, modeling spatio-temporal patterns of holistic functional brain networks plays an important role in understanding brain function. Compared to traditional modeling methods such as principal component analysis, independent component analysis, and sparse coding, superior performance has been achieved by recent deep learning methodologies. However, there are still two limitations of existing deep learning approaches for functional brain network modeling. They either (1) merely modeled a single targeted network and ignored holistic ones at one time, or (2) underutilized both spatial and temporal features of fMRI during network modeling, and the spatial/temporal accuracy was thus not warranted. To address these limitations, we proposed a novel Multi-Head Guided Attention Graph Neural Network (Multi-Head GAGNN) to simultaneously model both spatial and temporal patterns of holistic functional brain networks. Specifically, a spatial Multi-Head Attention Graph U-Net was first adopted to model the spatial patterns of multiple brain networks, and a temporal Multi-Head Guided Attention Network was then introduced to model the corresponding temporal patterns under the guidance of modeled spatial patterns. Based on seven task fMRI datasets from the public Human Connectome Project and resting state fMRI datasets from the public Autism Brain Imaging Data Exchange I of 1448 subjects, the proposed Multi-Head GAGNN showed superior ability and generalizability in modeling both spatial and temporal patterns of holistic functional brain networks in individual brains compared to other state-of-the-art (SOTA) models. Furthermore, the modeled spatio-temporal patterns of functional brain networks via the proposed Multi-Head GAGNN can better predict the individual cognitive behavioral measures compared to the other SOTA models. This study provided a novel and powerful tool for brain function modeling as well as for understanding the brain-cognitive behavior associations.

Spatiotemporal Graph Neural Network Modelling Perfusion MRI

Predicting isocitrate dehydrogenase mutation status in glioma using structural brain networks and graph neural networks

A Deep Graph Neural Network Architecture for Modelling Spatio-temporal Dynamics in resting-state functional MRI Data

A Guided Attention 4D Convolutional Neural Network for Modeling Spatio-Temporal Patterns of Functional Brain Networks.

Modeling spatio-temporal patterns of holistic functional brain networks via multi-head guided attention graph neural networks (Multi-Head GAGNNs)

SGPNet: A Three-Dimensional Multitask Residual Framework for Segmentation and IDH Genotype Prediction of Gliomas

Perfusion parameter map generation from TOF-MRA in stroke using generative adversarial networks

Multi-class Brain Tumor Segmentation using Graph Attention Network

PathoGraph: An Attention-Based Graph Neural Network Capable of Prognostication Based on CD276 Labelling of Malignant Glioma Cells

Treatment-aware Diffusion Probabilistic Model for Longitudinal MRI Generation and Diffuse Glioma Growth Prediction

Quantitative perfusion maps using a novelty spatiotemporal convolutional neural network

Diagnosis of Glioblastoma Multiforme Progression Via Interpretable Structure-Constrained Graph Neural Networks.

3D Graph Attention Networks for High Fidelity Pediatric Glioma Segmentation

Glioma Growth Prediction via Generative Adversarial Learning from Multi-Time Points Magnetic Resonance Images

Spatial Temporal Graph Convolution with Graph Structure Self-learning for Early MCI Detection

Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?

Image-to-image generative adversarial networks for synthesizing perfusion parameter maps from DSC-MR images in cerebrovascular disease

STNAGNN: Spatiotemporal Node Attention Graph Neural Network for Task-based fMRI Analysis

Personalized Predictions of Glioblastoma Infiltration: Mathematical Models, Physics-Informed Neural Networks and Multimodal Scans

GraphMriNet: a few-shot brain tumor MRI image classification model based on Prewitt operator and graph isomorphic network

MMMNA-Net for Overall Survival Time Prediction of Brain Tumor Patients