Abstract:Functional magnetic resonance imaging (fMRI) is a methodology for measuring human brain activities. It has become more and more popular in neural decoding due to its noninvasive. Neural decoding aims to establishing models to reconstruct external stimuli or features of stimuli from known brain responses, so that we can understand the principles of brain functions such as emotion, cognition and language. Neural decoding based on fMRI is of great significance for further understanding the mechanism of brain operation. Most existing studies take multi-scale topology information of brain networks obtained from fMRI into account in neural decoding. However, they always ignore the simultaneous modeling of network structure and hemodynamic response, thus leading to information loss. In addition, current multi-scale methods usually only utilize spatial or logical reasoning relationship of brain networks, which brings challenge to precise neural decoding. In this work, we present a novel and robust multi-scale spatial and logical reasoning learning framework (MSLR) for fMRI-based neural decoding. Specifically, we first design graph signal wavelet generation module to combine brain network topology and node information to construct multi-scale representation of brain networks in a local to global manner. Then, we develop multi-scale information fusion module that can simultaneously model the spatial and logical reasoning relationship of brain networks, it can also learn discriminative multi-scale features with brain state transition. Finally, we construct a neural decoding module to predict the brain states. We evaluated the framework on the public Human Connectome Project (HCP) dataset that included 986 participants. The experimental results with support vector machine (SVM) outperform current state-of-the-art methods on four evaluation metrics (accuracy: 91.58, kappa coefficient: 0.883, macro F1: 0.865 and hamming distance: 0.105) under 19 different stimuli spanning 7 different cognitive tasks. The interpretation of the learned multi-scale representation replicates neuroscientific findings from previous fMRI studies and renews the multi-scale information flow pattern of brain network in neural decoding.

Simultaneous Spatial-Temporal Decomposition for Connectome-Scale Brain Networks by Deep Sparse Recurrent Auto-Encoder

Evolutional Neural Architecture Search for Optimization of Spatiotemporal Brain Network Decomposition.

Neural Architecture Search for Optimization of Spatial-Temporal Brain Network Decomposition

Exploring Brain Hemodynamic Response Patterns Via Deep Recurrent Autoencoder

Modeling Task Fmri Data Via Deep Convolutional Autoencoder

DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks

Constructing Fine-Granularity Functional Brain Network Atlases Via Deep Convolutional Autoencoder.

Sparse Deep Neural Network for Encoding and Decoding the Structural Connectome

Modeling and augmenting of fMRI data using deep recurrent variational auto-encoder

Encoding Multi-Resolution Brain Networks Using Unsupervised Deep Learning

Sequential sparse autoencoder for dynamic heading representation in ventral intraparietal area

SS-Net: 3D Spatial-Spectral Network for Cerebrovascular Segmentation in TOF-MRA

3d Convolutional Long-Short Term Memory Network For Spatiotemporal Modeling Of Fmri Data

A Generic Framework for Embedding Human Brain Function with Temporally Correlated Autoencoder.

Multi-Scale Spatio-Temporal Fusion with Adaptive Brain Topology Learning for Fmri Based Neural Decoding.

Deep multivariate autoencoder for capturing complexity in Brain Structure and Behaviour Relationships

Deep Representations for Time-varying Brain Datasets

Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics

Jointly Fusing Multi-Scale Spatial-Logical Brain Networks: A Neural Decoding Method

Spatio-temporal Modeling of Connectome-Scale Brain Network Interactions Via Time-Evolving Graphs

Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis