Abstract:Multimodal neuroimaging modeling has become a widely used approach but confronts considerable challenges due to heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret these diverse datasets within a cohesive analytical framework. In our research, we amalgamate functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), and structural MRI (sMRI) into a cohesive framework. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging-derived features from various modalities-functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI-within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. This technique enhances interpretability at connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved predictive accuracy and uncovers crucial anatomical features and essential neural connections, deepening our understanding of brain structure and function. This study not only advances multi-modal neuroimaging analytics by offering a novel method for the integrated analysis of diverse imaging modalities but also improves the understanding of intricate relationship between the brain's structural and functional networks and cognitive development.

Multiscale and multimodal fusion of tract-tracing and DTI-derived fibers in macaque brains

Multi-scale and Multimodal Fusion of Tract-tracing, Myelin Stain and DTI-derived Fibers in Macaque Brains

Optimization of Macaque Brain DMRI Connectome by Neuron Tracing and Myelin Stain Data

Construct and assess multimodal mouse brain connectomes via joint modeling of multi-scale DTI and neuron tracer data.

Joint Representation of Connectome-Scale Structural and Functional Profiles for Identification of Consistent Cortical Landmarks in Macaque Brain

Optimization of Large-Scale Mouse Brain Connectome Via Joint Evaluation of DTI and Neuron Tracing Data.

A Large-Scale High-Density Weighted Structural Connectome of the Macaque Brain Acquired by Predicting Missing Links.

Imaging the structural connectome with hybrid diffusion MRI-microscopy tractography

Mapping brain-wide excitatory projectome of primate prefrontal cortex at submicron resolution and comparison with diffusion tractography

Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

Multi-way Regression Reveals Backbone of Macaque Structural Brain Connectivity in Longitudinal Datasets.

Connectome-scale Functional Intrinsic Connectivity Networks in Macaques.

Species-Preserved Structural Connections Revealed By Sparse Tensor Cca

Species Preserved and Exclusive Structural Connections Revealed by Sparse CCA.

Anatomy-Guided Discovery of Large-Scale Consistent Connectivity-Based Cortical Landmarks

Unifying the analyses of anatomical and diffusion tensor images using volume-preserved warping.

Constructing fiber atlases for functional ROIs via fMRI-Guided DTI image registration

Functional Informed Fiber Tracking Using Combination of Diffusion and Functional MRI.

Fusing DTI and fMRI data: A survey of methods and applications

Brain connectomes come of age