Abstract:Structural magnetic resonance imaging (sMRI) reveals the structural organization of the brain. Learning general brain representations from sMRI is an enduring topic in neuroscience. Previous deep learning models neglect that the brain, as the core of cognition, is distinct from other organs whose primary attribute is anatomy. Capturing the high-level representation associated with inter-individual cognitive variability is key to appropriately represent the brain. Given that this cognition-related information is subtle, mixed, and distributed in the brain structure, sMRI-based models need to both capture fine-grained details and understand how they relate to the overall global structure. Additionally, it is also necessary to explicitly express the cognitive information that implicitly embedded in local-global image features. Therefore, we propose MCPATS, a brain representation learning framework that combines Multi-task Collaborative Pre-training (MCP) and Adaptive Token Selection (ATS). First, we develop MCP, including mask-reconstruction to understand global context, distort-restoration to capture fine-grained local details, adversarial learning to integrate features at different granularities, and age-prediction, using age as a surrogate for cognition to explicitly encode cognition-related information from local-global image features. This co-training allows progressive learning of implicit and explicit cognition-related representations. Then, we develop ATS based on mutual attention for downstream use of the learned representation. During fine-tuning, the ATS highlights discriminative features and reduces the impact of irrelevant information. MCPATS was validated on three different public datasets for brain disease diagnosis, outperforming competing methods and achieving accurate diagnosis. Further, we performed detailed analysis to confirm that the MCPATS-learned representation captures cognition-related information.

Wills Aligner: A Robust Multi-Subject Brain Representation Learner

Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding

Aligning brain functions boosts the decoding of visual semantics in novel subjects

See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI

Through their eyes: multi-subject Brain Decoding with simple alignment techniques

MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding

Multi-subject brain decoding with multi-task feature selection.

Achieving More Human Brain-Like Vision via Human EEG Representational Alignment

A Subject-Adaptive Brain State Decoding model via Ensemble Transfer Learning

Multi-Task Collaborative Pre-Training and Adaptive Token Selection: A Unified Framework for Brain Representation Learning

Lite-Mind: Towards Efficient and Robust Brain Representation Learning

Mapping EEG Signals to Visual Stimuli: A Deep Learning Approach to Match Vs. Mismatch Classification

MindBridge: A Cross-Subject Brain Decoding Framework

Neural Latent Aligner: Cross-trial Alignment for Learning Representations of Complex, Naturalistic Neural Data

Brainnetformer: Decoding Brain Cognitive States with Spatial-Temporal Cross Attention

Transfer Learning for Brain-Computer Interfaces: A Euclidean Space Data Alignment Approach.

BLSAN: A Brain Lateralization-Guided Subject Adaptive Network for Motor Imagery Classification

Multi-task Collaborative Pre-training and Individual-adaptive-tokens Fine-tuning: A Unified Framework for Brain Representation Learning

ALIGNING BRAINS INTO A SHARED SPACE IMPROVES THEIR ALIGNMENT TO LARGE LANGUAGE MODELS

Lite-Mind: Towards Efficient and Robust Brain Representation Network