Abstract:Brain decoding, a pivotal field in neuroscience, aims to reconstruct stimuli from acquired brain signals, primarily utilizing functional magnetic resonance imaging (fMRI). Currently, brain decoding is confined to a per-subject-per-model paradigm, limiting its applicability to the same individual for whom the decoding model is trained. This constraint stems from three key challenges: 1) the inherent variability in input dimensions across subjects due to differences in brain size; 2) the unique intrinsic neural patterns, influencing how different individuals perceive and process sensory information; 3) limited data availability for new subjects in real-world scenarios hampers the performance of decoding models. In this paper, we present a novel approach, MindBridge, that achieves cross-subject brain decoding by employing only one model. Our proposed framework establishes a generic paradigm capable of addressing these challenges by introducing biological-inspired aggregation function and novel cyclic fMRI reconstruction mechanism for subject-invariant representation learning. Notably, by cycle reconstruction of fMRI, MindBridge can enable novel fMRI synthesis, which also can serve as pseudo data augmentation. Within the framework, we also devise a novel reset-tuning method for adapting a pretrained model to a new subject. Experimental results demonstrate MindBridge's ability to reconstruct images for multiple subjects, which is competitive with dedicated subject-specific models. Furthermore, with limited data for a new subject, we achieve a high level of decoding accuracy, surpassing that of subject-specific models. This advancement in cross-subject brain decoding suggests promising directions for wider applications in neuroscience and indicates potential for more efficient utilization of limited fMRI data in real-world scenarios. Project page: https://littlepure2333.github.io/MindBridge

Decoding Brain States From fMRI Signals by Using Unsupervised Domain Adaptation

Brain Decoding Using Deep Convolutional Network and Its Application in Cross-Subject Analysis.

Transfer Learning-Based Behavioural Task Decoding from Brain Activity

Functional annotation of human cognitive states using deep graph convolution

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

Attention module improves both performance and interpretability of four‐dimensional functional magnetic resonance imaging decoding neural network

An Adaptively Weighted Averaging Method for Regional Time Series Extraction of fMRI-based Brain Decoding

Decoding Behavior Tasks From Brain Activity Using Deep Transfer Learning

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

Volume-Wise Decoding of Human Brain's Task fMRI via Deep Learning

Deep learning of fMRI big data: a novel approach to subject-transfer decoding

MindBridge: A Cross-Subject Brain Decoding Framework

Decoding and mapping task states of the human brain via deep learning

Decoding Task Sub-type States with Group Deep Bidirectional Recurrent Neural Network

Multi-Scale Spatio-Temporal Fusion with Adaptive Brain Topology Learning for Fmri Based Neural Decoding.

Attention module improves both performance and interpretability of 4D fMRI decoding neural network

Task sub-type states decoding via group deep bidirectional recurrent neural network

Source-Free Collaborative Domain Adaptation via Multi-Perspective Feature Enrichment for Functional MRI Analysis

Brain Decoding Using fMRI Images for Multiple Subjects through Deep Learning

Task State Decoding and Mapping of Individual Four-Dimensional Fmri Time Series Using Deep Neural Network