Abstract:Functional magnetic resonance imaging (fMRI) is a powerful technique with the potential to estimate individual variations in behavioral and cognitive traits. Joint learning of multiple datasets can utilize their complementary information so as to improve learning performance, but it also gives rise to the challenge for data fusion to effectively integrate brain patterns elicited by multiple fMRI data. However, most of the current data fusion methods analyze each single dataset separately and further infer the relationship among them, which fail to utilize the multidimensional structure inherent across modalities and may ignore complex but important interactions. To address this issue, we propose a novel sparse tensor decomposition method to integrate multiple task-stimulus (paradigm) fMRI data. Seeing each paradigm fMRI as one modality, our proposed method considers the relationships across subjects and modalities simultaneously. In specific, a third-order tensor is first modeled by using the functional network connectivity (FNC) of subjects in multiple fMRI paradigms. A novel sparse tensor decomposition with the regularization terms is designed to factorize the tensor into a series of rank-one components, which can extract the shared components across modalities as the embedded features. The $L_{2,1}$-norm regularizer (i.e., group sparsity) is enforced to select a few common features among multiple subjects. Validation of the proposed method is performed on realistic three paradigm fMRI datasets from the Philadelphia Neurodevelopmental Cohort (PNC) study, for the study of the relationship between the FNC and human cognitive abilities. Experimental results show our method outperforms several other competing methods in the prediction of individuals with different cognitive behaviors via the wide range achievement test (WRAT). Furthermore, our method discovers the FNC related to the cognitive beha-iors, such as the connectivity associated with the default mode network (DMN) for three paradigms, and the connectivity between DMN and visual (VIS) domains within the emotion task.

A Novel Graph Wavelet Model for Brain Multi-scale Activational-Connectional Feature Fusion

Regional Brain Fusion: Graph Convolutional Network for Alzheimer's Disease Prediction and Analysis

Multi-graph Fusion for Functional Neuroimaging Biomarker Detection

MSTGC: Multi-Channel Spatio-Temporal Graph Convolution Network for Multi-Modal Brain Networks Fusion

Fusing Structural and Functional Connectivities using Disentangled VAE for Detecting MCI

A Graph-Based Feature Extraction Algorithm Towards a Robust Data Fusion Framework for Brain-Computer Interfaces

A Novel Multi-Atlas Fusion Model Based On Contrastive Learning For Functional Connectivity Graph Diagnosis.

Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

GSCFN: A graph self-construction and fusion network for semi-supervised brain tissue segmentation in MRI

MMF-NNs: Multi-modal Multi-granularity Fusion Neural Networks for brain networks and its application to epilepsy identification

Deep Fusion of Multi-Template using Spatio-Temporal Weighted Multi-Hypergraph Convolutional Networks for Brain Disease Analysis

Brain Functional Connectivity Analysis Based on Multi-Graph Fusion

Multi-Paradigm fMRI Fusion via Sparse Tensor Decomposition in Brain Functional Connectivity Study

A Heterogeneous Graph Neural Network Fusing Functional and Structural Connectivity for MCI Diagnosis

Simultaneous EEG-fMRI: Trial level spatio-temporal fusion for hierarchically reliable information discovery

A Heterogeneous Graph Based Framework for Multimodal Neuroimaging Fusion Learning

A Multi-Scale Fusion Convolutional Neural Network Based on Attention Mechanism for the Visualization Analysis of EEG Signals Decoding

Fmri-Based Brain Disease Diagnosis: A Graph Network Approach

Interpretable Multimodal Fusion Networks Reveal Mechanisms of Brain Cognition

BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis