Abstract:Multimodal neuroimaging modeling has become a widely used approach but confronts considerable challenges due to heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret these diverse datasets within a cohesive analytical framework. In our research, we amalgamate functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), and structural MRI (sMRI) into a cohesive framework. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging-derived features from various modalities-functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI-within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. This technique enhances interpretability at connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved predictive accuracy and uncovers crucial anatomical features and essential neural connections, deepening our understanding of brain structure and function. This study not only advances multi-modal neuroimaging analytics by offering a novel method for the integrated analysis of diverse imaging modalities but also improves the understanding of intricate relationship between the brain's structural and functional networks and cognitive development.

Joint representation of cortical folding, structural connectivity and functional networks

Joint Representation of Consistent Structural and Functional Profiles for Identification of Common Cortical Landmarks.

Joint Analysis of Cortical Folding Pattern, Thickness and Fiber Density Using Structural and DTI Data

Joint Representation of Connectome-Scale Structural and Functional Profiles for Identification of Consistent Cortical Landmarks in Macaque Brain

Anatomy-Guided Discovery of Large-Scale Consistent Connectivity-Based Cortical Landmarks

Exploring 3-Hinge Gyral Folding Patterns among HCP Q3 868 Human Subjects

Joint Modeling for Geometry and Functionality of Cerebral Cortical Surface Images

Predicting Cortical ROIs Via Joint Modeling of Anatomical and Connectional Profiles.

Identifying Cross-individual Correspondences of 3-hinge Gyri

Predicting Cortical 3-Hinge Locations Via Structural Connective Features

Assessing Regularity and Variability of Cortical Folding Patterns of Working Memory ROIs

Human Cortical Anatomical Networks Assessed by Structural Mri

Evolutionarily-preserved Consistent Gyral Folding Patterns Across Primate Brains

Cortex2vector: anatomical embedding of cortical folding patterns

A framework to improve the alignment of individual cytoarchitectonic maps of the Julich-Brain atlas using cortical folding landmarks

Exploring Intrinsic Functional Differences of Gyri, Sulci and 2-Hinge, 3-Hinge Joints on Cerebral Cortex.

Exploring Brain Networks via Structured Sparse Representation of fMRI Data.

Modeling Functional Dynamics of Cortical Gyri and Sulci.

Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

Multi-view Graph Matching of Cortical Landmarks