An Adaptively Weighted Averaging Method for Regional Time Series Extraction of fMRI-based Brain Decoding

Jianfei Zhu,Baichun Wei,Jiaru Tian,Feng Jiang,Chunzhi Yi

2024-07-11

Abstract:Brain decoding that classifies cognitive states using the functional fluctuations of the brain can provide insightful information for understanding the brain mechanisms of cognitive functions. Among the common procedures of decoding the brain cognitive states with functional magnetic resonance imaging (fMRI), extracting the time series of each brain region after brain parcellation traditionally averages across the voxels within a brain region. This neglects the spatial information among the voxels and the requirement of extracting information for the downstream tasks. In this study, we propose to use a fully connected neural network that is jointly trained with the brain decoder to perform an adaptively weighted average across the voxels within each brain region. We perform extensive evaluations by cognitive state decoding, manifold learning, and interpretability analysis on the Human Connectome Project (HCP) dataset. The performance comparison of the cognitive state decoding presents an accuracy increase of up to 5\% and stable accuracy improvement under different time window sizes, resampling sizes, and training data sizes. The results of manifold learning show that our method presents a considerable separability among cognitive states and basically excludes subject-specific information. The interpretability analysis shows that our method can identify reasonable brain regions corresponding to each cognitive state. Our study would aid the improvement of the basic pipeline of fMRI processing.

Human-Computer Interaction,Neurons and Cognition

What problem does this paper attempt to address?

This paper focuses on the problem of brain decoding based on functional magnetic resonance imaging (fMRI), especially on how to better extract the time series of brain regions. The traditional approach is to extract the regional time series by averaging the voxel signals within each brain region, but this method ignores the spatial information between voxels and the information demands of downstream tasks. The paper proposes an Adaptively Weighted Average Time Series (AWATS) method, which uses a fully connected neural network to weight the voxels within each brain region for average, in order to adaptively preserve the spatial information and combine it with downstream tasks (such as cognitive state decoding) for training. The researchers conducted extensive evaluations on the Human Connectome Project (HCP) dataset, including cognitive state decoding, manifold learning, and interpretability analysis. The results show that using the AWATS method can improve the accuracy, stability, and separability of cognitive state decoding, and can identify reasonable brain regions corresponding to various cognitive states. In addition, this method can exclude individual-specific information and enhance the interpretability of the model. The paper also mentions the limitations of existing methods, such as the use of traditional average time series (ATS) which may overlook the voxel differences and task-related information. To validate the effectiveness of adaptively preserving spatial information, the researchers designed a Spatial-temporal Deep fMRI (STDfMRI) model that combines AWATS with ATS, and demonstrated the advantages of AWATS by comparing the performance of the two input methods. In summary, this paper attempts to improve the basic process of fMRI data processing by optimizing the extraction of region time series through an adaptive weighted average method, thereby improving the accuracy and understanding of brain cognitive state decoding.

An Adaptively Weighted Averaging Method for Regional Time Series Extraction of fMRI-based Brain Decoding

Jointly Fusing Multi-Scale Spatial-Logical Brain Networks: A Neural Decoding Method

Multi-Scale Spatio-Temporal Fusion with Adaptive Brain Topology Learning for Fmri Based Neural Decoding.

Decoding Brain States From fMRI Signals by Using Unsupervised Domain Adaptation

Functional annotation of human cognitive states using deep graph convolution

Attention module improves both performance and interpretability of four‐dimensional functional magnetic resonance imaging decoding neural network

Volume-Wise Decoding of Human Brain's Task fMRI via Deep Learning

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

Brain decoding from functional MRI using long short-term memory recurrent neural networks

Brain Decoding Using Deep Convolutional Network and Its Application in Cross-Subject Analysis.

Decoding and mapping task states of the human brain via deep learning

Attention module improves both performance and interpretability of 4D fMRI decoding neural network

Decoding Digital Visual Stimulation From Neural Manifold With Fuzzy Leaning on Cortical Oscillatory Dynamics

Transfer Learning-Based Behavioural Task Decoding from Brain Activity

Improved Application of Sparse Representation Classifier in fMRI-based Brain State Decoding.

A Novel Brain Decoding Method: A Correlation Network Framework for Revealing Brain Connections

Decoding brain states using functional magnetic resonance imaging

Brain State Decoding Based on Fmri Using Semisupervised Sparse Representation Classifications.

Task State Decoding and Mapping of Individual Four-Dimensional Fmri Time Series Using Deep Neural Network

Brainnetformer: Decoding Brain Cognitive States with Spatial-Temporal Cross Attention

Sparse Deep Neural Network for Encoding and Decoding the Structural Connectome