Abstract:Decoding brain states under different cognitive tasks from functional magnetic resonance imaging (fMRI) data has attracted great attention in the neuroimaging filed. However, the well-known temporal dependency in fMRI sequences has not been fully exploited in existing studies, due to the limited temporal-modeling capacity of the backbone machine learning algorithms and rigid training sample organization strategies upon which the brain decoding methods are built. To address these limitations, we propose a novel method for fine-grain brain state decoding, namely, group deep bidirectional recurrent neural network (Group-DBRNN) model. We first propose a training sample organization strategy that consists of a group-task sample generation module and a multiple-scale random fragment strategy (MRFS) module to collect training samples that contain rich task-relevant brain activity contrast (i.e., the comparison of neural activity patterns between different tasks) and maintain the temporal dependency. We then develop a novel decoding model by replacing the unidirectional RNNs that are widely used in existing brain state decoding studies with bidirectional stacked RNNs to better capture the temporal dependency, and by introducing a multi-task interaction layer (MTIL) module to effectively model the task-relevant brain activity contrast. Our experimental results on the Human Connectome Project task fMRI dataset (7 tasks consisting of 23 task sub-type states) show that the proposed model achieves an average decoding accuracy of 94.7% over the 23 fine-grain sub-type states. Meanwhile, our extensive interpretations of the intermediate features learned in the proposed model via visualizations and quantitative assessments of their discriminability and inter-subject alignment evidence that the proposed model can effectively capture the temporal dependency and task-relevant contrast.

Decoding Task Sub-type States with Group Deep Bidirectional Recurrent Neural Network

Task sub-type states decoding via group deep bidirectional recurrent neural network

Brain decoding from functional MRI using long short-term memory recurrent neural networks

Task State Decoding and Mapping of Individual Four-Dimensional Fmri Time Series Using Deep Neural Network

Decoding Behavior Tasks From Brain Activity Using Deep Transfer Learning

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

Decoding and mapping task states of the human brain via deep learning

Decoding Brain States From fMRI Signals by Using Unsupervised Domain Adaptation

Supervised Brain Network Learning Based on Deep Recurrent Neural Networks.

Design of Deep Learning Model for Task-Evoked fMRI Data Classification

Attention module improves both performance and interpretability of four‐dimensional functional magnetic resonance imaging decoding neural network

Volume-Wise Decoding of Human Brain's Task fMRI via Deep Learning

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

Identifying Brain Networks at Multiple Time Scales via Deep Recurrent Neural Network

A Pilot Study on Two Stage Decoding Strategies.

Exploring Brain Hemodynamic Response Patterns Via Deep Recurrent Autoencoder

Recognizing Brain States Using Deep Sparse Recurrent Neural Network

Decoding task specific and task general functional architectures of the brain

Task Fmri Data Analysis Based on Supervised Stochastic Coordinate Coding.

Group-level Brain Decoding with Deep Learning

Deep learning of fMRI big data: a novel approach to subject-transfer decoding