Abstract:Functional magnetic resonance imaging (fMRI) is a methodology for measuring human brain activities. It has become more and more popular in neural decoding due to its noninvasive. Neural decoding aims to establishing models to reconstruct external stimuli or features of stimuli from known brain responses, so that we can understand the principles of brain functions such as emotion, cognition and language. Neural decoding based on fMRI is of great significance for further understanding the mechanism of brain operation. Most existing studies take multi-scale topology information of brain networks obtained from fMRI into account in neural decoding. However, they always ignore the simultaneous modeling of network structure and hemodynamic response, thus leading to information loss. In addition, current multi-scale methods usually only utilize spatial or logical reasoning relationship of brain networks, which brings challenge to precise neural decoding. In this work, we present a novel and robust multi-scale spatial and logical reasoning learning framework (MSLR) for fMRI-based neural decoding. Specifically, we first design graph signal wavelet generation module to combine brain network topology and node information to construct multi-scale representation of brain networks in a local to global manner. Then, we develop multi-scale information fusion module that can simultaneously model the spatial and logical reasoning relationship of brain networks, it can also learn discriminative multi-scale features with brain state transition. Finally, we construct a neural decoding module to predict the brain states. We evaluated the framework on the public Human Connectome Project (HCP) dataset that included 986 participants. The experimental results with support vector machine (SVM) outperform current state-of-the-art methods on four evaluation metrics (accuracy: 91.58, kappa coefficient: 0.883, macro F1: 0.865 and hamming distance: 0.105) under 19 different stimuli spanning 7 different cognitive tasks. The interpretation of the learned multi-scale representation replicates neuroscientific findings from previous fMRI studies and renews the multi-scale information flow pattern of brain network in neural decoding.

A Joint Analysis of Multi-Paradigm fMRI Data With Its Application to Cognitive Study

Multi-Paradigm fMRI Fusion via Sparse Tensor Decomposition in Brain Functional Connectivity Study

Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

Task Fmri Data Analysis Based on Supervised Stochastic Coordinate Coding.

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

An Explainable Autoencoder with Multi-Paradigm Fmri Fusion for Identifying Differences in Dynamic Functional Connectivity During Brain Development

Jointly Fusing Multi-Scale Spatial-Logical Brain Networks: A Neural Decoding Method

A non-parametric approach for co-analysis of multi-modal brain imaging data: Application to Alzheimer's disease

Analysis of Longitudinal Change Patterns in Developing Brain Using Functional and Structural Magnetic Resonance Imaging via Multimodal Fusion

Multi-Task Collaborative Pre-Training and Adaptive Token Selection: A Unified Framework for Brain Representation Learning

Multi-task Collaborative Pre-training and Individual-adaptive-tokens Fine-tuning: A Unified Framework for Brain Representation Learning

Modeling Cognitive Processes Via Multi-stage Consistent Functional Response Detection.

Brain structure–function associations identified in large-scale neuroimaging data

Multi-source brain computing with systematic fusion for smart health

Integrating Multimodal Neuroimaging and Genetics: A Structurally-Linked Sparse Canonical Correlation Analysis Approach

Source-Free Collaborative Domain Adaptation via Multi-Perspective Feature Enrichment for Functional MRI Analysis

Multi-graph Fusion for Functional Neuroimaging Biomarker Detection

Systematic Fusion of Multi-Source Cognitive Networks with Graph Learning - A Study on Fronto-Parietal Network

Cross-subject Brain Functional Connectivity Analysis for Multi-task Cognitive State Evaluation

Analysis of FMRI data using an integrated principal component analysis and supervised affinity propagation clustering approach.