Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

Gang Qu,Ziyu Zhou,Vince D. Calhoun,Aiying Zhang,Yu-Ping Wang
2024-08-26
Abstract:Multimodal neuroimaging modeling has becomes a widely used approach but confronts considerable challenges due to heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret these diverse datasets within a cohesive analytical framework. In our research, we amalgamate functional magnetic resonance imaging, diffusion tensor imaging, and structural MRI into a cohesive framework. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging derived features from various modalities: functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. This technique enhances interpretability at connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved predictive accuracy and uncovers crucial anatomical features and essential neural connections, deepening our understanding of brain structure and function.
Neurons and Cognition,Machine Learning
What problem does this paper attempt to address?
This paper aims to address the challenges of data heterogeneity and integration in multimodal neuroimaging. Specifically, researchers attempt to provide a comprehensive framework by combining functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), and structural magnetic resonance imaging (sMRI) to fully understand the connectivity and anatomical features of the brain. These different imaging modalities each provide unique information about different aspects of the brain, but there are significant differences in their data types, scales, and formats, which pose difficulties for data integration. Therefore, the main research question is: **How can multimodal neuroimaging data from fMRI, DTI, and sMRI be effectively integrated and interpreted to provide a deeper understanding of brain structure and functional networks and their relationship with cognitive development?** To meet this challenge, researchers propose a method based on Interpretable Graph Neural Networks (IGNN). This method uses a unified anatomical atlas (such as the Glasser atlas) to standardize data from different modalities and integrates and analyzes this data through a graph neural network model. This method can not only overcome the challenge of data heterogeneity but also improve the interpretability of the model, thereby better revealing the complex relationships between brain structure and function and their roles in cognitive development.