Abstract:Resting-state functional magnetic resonance imaging (rs-fMRI) is a noninvasive technique pivotal for understanding human neural mechanisms of intricate cognitive processes. Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest, or dynamic functional connectivity matrices with a sliding window approach. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. While deep learning has gained substantial popularity for modeling complex relational data, its application to uncovering the spatiotemporal dynamics of the brain is still limited. We propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series and employs a specialized graph neural network for the final classification. Our model, DSAM, leverages temporal causal convolutional networks to capture the temporal dynamics in both low- and high-level feature representations, a temporal attention unit to identify important time points, a self-attention unit to construct the goal-specific connectivity matrix, and a novel variant of graph neural network to capture the spatial dynamics for downstream classification. To validate our approach, we conducted experiments on the Human Connectome Project dataset with 1075 samples to build and interpret the model for the classification of sex group, and the Adolescent Brain Cognitive Development Dataset with 8520 samples for independent testing. Compared our proposed framework with other state-of-art models, results suggested this novel approach goes beyond the assumption of a fixed connectivity matrix and provides evidence of goal-specific brain connectivity patterns, which opens up the potential to gain deeper insights into how the human brain adapts its functional connectivity specific to the task at hand.

Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis

DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks

Modeling Task Fmri Data Via Deep Convolutional Autoencoder

State-space model with deep learning for functional dynamics estimation in resting-state fMRI

Spatial-Temporal Convolutional Attention for Mapping Functional Brain Networks.

3d Convolutional Long-Short Term Memory Network For Spatiotemporal Modeling Of Fmri Data

MambaRecon: MRI Reconstruction with Structured State Space Models

Multi-head attention-based masked sequence model for mapping functional brain networks

Identify Hierarchical Structures from Task-Based fMRI Data via Hybrid Spatiotemporal Neural Architecture Search Net

Hierarchical Organization of Functional Brain Networks Revealed by Hybrid Spatiotemporal Deep Learning

Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN)

Multi-Scale Spatio-Temporal Fusion with Adaptive Brain Topology Learning for Fmri Based Neural Decoding.

Exploring Brain Effective Connectivity Networks Through Spatiotemporal Graph Convolutional Models.

Hierarchical Extraction of Functional Connectivity Components in Human Brain Using Resting-State fMRI

Spatio-temporal Modeling of Connectome-Scale Brain Network Interactions Via Time-Evolving Graphs

Spatial-Temporal Mamba Network for EEG-based Motor Imagery Classification

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

A Deep Graph Neural Network Architecture for Modelling Spatio-temporal Dynamics in resting-state functional MRI Data

Simultaneous Spatial-Temporal Decomposition for Connectome-Scale Brain Networks by Deep Sparse Recurrent Auto-Encoder

Modeling Hierarchical Brain Networks Via Volumetric Sparse Deep Belief Network

A Guided Attention 4D Convolutional Neural Network for Modeling Spatio-Temporal Patterns of Functional Brain Networks.