Abstract:Multimodal neuroimaging modeling has become a widely used approach but confronts considerable challenges due to heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret these diverse datasets within a cohesive analytical framework. In our research, we amalgamate functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), and structural MRI (sMRI) into a cohesive framework. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging-derived features from various modalities-functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI-within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. This technique enhances interpretability at connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved predictive accuracy and uncovers crucial anatomical features and essential neural connections, deepening our understanding of brain structure and function. This study not only advances multi-modal neuroimaging analytics by offering a novel method for the integrated analysis of diverse imaging modalities but also improves the understanding of intricate relationship between the brain's structural and functional networks and cognitive development.

Integration of Multimodal Data for Deciphering Brain Disorders.

Multimodal Magnetic Resonance Imaging for Brain Disorders: Advances and Perspectives

Incomplete Multimodal Learning for Complex Brain Disorders Prediction

Multimodal Fusion of Brain Imaging Data: Methods and Applications

Multimodal neuroimaging computing: a review of the applications in neuropsychiatric disorders

A review of heterogeneous data mining for brain disorders

Big Connectome Imaging Data in Psychiatric Disorders

Deep Multimodal Predictome for Studying Mental Disorders

Deciphering the Genetic Architecture of Human Brain Structure and Function: a Brief Survey on Recent Advances of Neuroimaging Genomics

Transcriptomic and neuroimaging data integration enhances machine learning classification of schizophrenia

Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

Integration of Multiple Genomic Imaging Data for the Study of Schizophrenia Using Joint Nonnegative Matrix Factorization.

Interpretable Multimodal Fusion Networks Reveal Mechanisms of Brain Cognition

Multimodal Neuroimaging Data Integration and Pathway Analysis

Integration of Imaging (epi)genomics Data for the Study of Schizophrenia Using Group Sparse Joint Nonnegative Matrix Factorization

Mapping Multi-Modal Brain Connectome for Brain Disorder Diagnosis Via Cross-Modal Mutual Learning.

Identification of multimodal brain imaging biomarkers in first-episode drugs-naive major depressive disorder through a multi-site large-scale MRI consortium data

Identifying disease sensitive and quantitative trait-relevant biomarkers from multidimensional heterogeneous imaging genetics data via sparse multimodal multitask learning.

Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation

The power of many brains: Catalyzing neuropsychiatric discovery through open neuroimaging data and large-scale collaboration