Abstract:Decoding brain states under different cognitive tasks from functional magnetic resonance imaging (fMRI) data has attracted great attention in the neuroimaging filed. However, the well-known temporal dependency in fMRI sequences has not been fully exploited in existing studies, due to the limited temporal-modeling capacity of the backbone machine learning algorithms and rigid training sample organization strategies upon which the brain decoding methods are built. To address these limitations, we propose a novel method for fine-grain brain state decoding, namely, group deep bidirectional recurrent neural network (Group-DBRNN) model. We first propose a training sample organization strategy that consists of a group-task sample generation module and a multiple-scale random fragment strategy (MRFS) module to collect training samples that contain rich task-relevant brain activity contrast (i.e., the comparison of neural activity patterns between different tasks) and maintain the temporal dependency. We then develop a novel decoding model by replacing the unidirectional RNNs that are widely used in existing brain state decoding studies with bidirectional stacked RNNs to better capture the temporal dependency, and by introducing a multi-task interaction layer (MTIL) module to effectively model the task-relevant brain activity contrast. Our experimental results on the Human Connectome Project task fMRI dataset (7 tasks consisting of 23 task sub-type states) show that the proposed model achieves an average decoding accuracy of 94.7% over the 23 fine-grain sub-type states. Meanwhile, our extensive interpretations of the intermediate features learned in the proposed model via visualizations and quantitative assessments of their discriminability and inter-subject alignment evidence that the proposed model can effectively capture the temporal dependency and task-relevant contrast.

Task State Decoding and Mapping of Individual Four-Dimensional Fmri Time Series Using Deep Neural Network

Decoding and mapping task states of the human brain via deep learning

Task sub-type states decoding via group deep bidirectional recurrent neural network

Decoding Task Sub-type States with Group Deep Bidirectional Recurrent Neural Network

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

Attention module improves both performance and interpretability of four‐dimensional functional magnetic resonance imaging decoding neural network

Volume-Wise Decoding of Human Brain's Task fMRI via Deep Learning

Design of Deep Learning Model for Task-Evoked fMRI Data Classification

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

Decoding Behavior Tasks From Brain Activity Using Deep Transfer Learning

Brain Decoding Using Deep Convolutional Network and Its Application in Cross-Subject Analysis.

Task Fmri Data Analysis Based on Supervised Stochastic Coordinate Coding.

Characterizing Functional Brain Networks Via Spatio-Temporal Attention 4D Convolutional Neural Networks (Sta-4Dcnns)

Attention module improves both performance and interpretability of 4D fMRI decoding neural network

Classification of Task-State fMRI Data Based on Circle-EMD and Machine Learning

Decoding Brain States From fMRI Signals by Using Unsupervised Domain Adaptation

Transfer Learning-Based Behavioural Task Decoding from Brain Activity

Characterizing And Differentiating Task-Based And Resting State Fmri Signals Via Two-Stage Dictionary Learning

Brain decoding from functional MRI using long short-term memory recurrent neural networks

Identifying Brain Networks at Multiple Time Scales via Deep Recurrent Neural Network

Decoding task specific and task general functional architectures of the brain