Abstract:We propose using subject embeddings to learn and exploit the structure in between‐subject variability in MEG decoding. We show that the combination of deep learning and subject embedding is crucial to closing the performance gap between subject‐ and group‐level models, with group models outperforming subject models on low‐accuracy subjects. Decoding brain imaging data are gaining popularity, with applications in brain‐computer interfaces and the study of neural representations. Decoding is typically subject‐specific and does not generalise well over subjects, due to high amounts of between subject variability. Techniques that overcome this will not only provide richer neuroscientific insights but also make it possible for group‐level models to outperform subject‐specific models. Here, we propose a method that uses subject embedding, analogous to word embedding in natural language processing, to learn and exploit the structure in between‐subject variability as part of a decoding model, our adaptation of the WaveNet architecture for classification. We apply this to magnetoencephalography data, where 15 subjects viewed 118 different images, with 30 examples per image; to classify images using the entire 1 s window following image presentation. We show that the combination of deep learning and subject embedding is crucial to closing the performance gap between subject‐ and group‐level decoding models. Importantly, group models outperform subject models on low‐accuracy subjects (although slightly impair high‐accuracy subjects) and can be helpful for initialising subject models. While we have not generally found group‐level models to perform better than subject‐level models, the performance of group modelling is expected to be even higher with bigger datasets. In order to provide physiological interpretation at the group level, we make use of permutation feature importance. This provides insights into the spatiotemporal and spectral information encoded in the models. All code is available on GitHub (https://github.com/ricsinaruto/MEG-group-decode).

Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding

Through their eyes: multi-subject Brain Decoding with simple alignment techniques

Aligning brain functions boosts the decoding of visual semantics in novel subjects

MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding

See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI

MindBridge: A Cross-Subject Brain Decoding Framework

Joint fMRI Decoding and Encoding with Latent Embedding Alignment

Enhancing Cross-Subject Fmri-to-video Decoding with Global-Local Functional Alignment

MindTuner: Cross-Subject Visual Decoding with Visual Fingerprint and Semantic Correction

Brain Dialogue Interface (BDI): A User-Friendly fMRI Model for Interactive Brain Decoding

Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction

Decoding Visual Experience and Mapping Semantics through Whole-Brain Analysis Using fMRI Foundation Models

From Sight to Insight: A Multi-task Approach with the Visual Language Decoding Model

Toward Generalizing Visual Brain Decoding to Unseen Subjects

Functional Alignment-Auxiliary Generative Adversarial Network-Based Visual Stimuli Reconstruction via Multi-Subject fMRI

Group-level Brain Decoding with Deep Learning

Group‐level brain decoding with deep learning

Decoding human brain activity with deep learning

Graph-Based Decoding Model for Functional Alignment of Unaligned fMRI Data

Decoding Behavior Tasks From Brain Activity Using Deep Transfer Learning