Abstract:Decoding brain imaging data are gaining popularity, with applications in brain-computer interfaces and the study of neural representations. Decoding is typicallysubject-specific and does not generalise well over subjects, due to high amounts ofbetween subject variability. Techniques that overcome this will not only providericher neuroscientific insights but also make it possible for group-level models to out-perform subject-specific models. Here, we propose a method that uses subjectembedding, analogous to word embedding in natural language processing, to learnand exploit the structure in between-subject variability as part of a decoding model,our adaptation of the WaveNet architecture for classification. We apply this to mag-netoencephalography data, where 15 subjects viewed 118 different images, with30 examples per image; to classify images using the entire 1 s window followingimage presentation. We show that the combination of deep learning and subjectembedding is crucial to closing the performance gap between subject- and group-level decoding models. Importantly, group models outperform subject models onlow-accuracy subjects (although slightly impair high-accuracy subjects) and can behelpful for initialising subject models. While we have not generally found group-levelmodels to perform better than subject-level models, the performance of groupmodelling is expected to be even higher with bigger datasets. In order to providephysiological interpretation at the group level, we make use of permutation featureimportance. This provides insights into the spatiotemporal and spectral informationencoded in the models. All code is available on GitHub (<a class="link-external link-https" href="https://github.com/ricsinaruto/MEG-group-decode" rel="external noopener nofollow">this https URL</a>).

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to overcome the challenge of inter - subject variability in brain - imaging data decoding in order to achieve a cross - subject (group - level) brain - activity decoding model. Specifically: 1. **Limitations of Subject - Specific Models**: Traditional brain - imaging decoding methods usually train a model for each subject separately. Although this method can well adapt to individual differences, it cannot use cross - subject data to improve the generalization ability of the model. In addition, each subject needs to train a model separately, which increases the computational burden and is prone to overfitting. 2. **Challenges of Cross - Subject Models**: Directly combining the data of multiple subjects to train a shared model (i.e., the "naive" group model) often performs poorly because the inter - subject variability is high, making it difficult for the model to capture effective features. 3. **Introducing Subject Embedding**: To overcome the above problems, this paper proposes a new method, which is to learn and utilize the structured variability between subjects by introducing subject embedding. Subject embedding is similar to word embedding in natural language processing and can capture the specific features of each subject, thereby improving the decoding performance in cross - subject models. 4. **Model Architecture**: The paper uses an improved WaveNet architecture for classification. Combined with subject embedding, it can improve the generalization ability of the model while maintaining individual differences. 5. **Experimental Verification**: By conducting experiments on MEG data of 15 subjects, the paper shows that after introducing subject embedding, the performance of the cross - subject model is significantly improved, especially more obvious on low - precision subjects. In addition, by fine - tuning the cross - subject model, the decoding performance can be further improved at the individual level. In conclusion, the main goal of this paper is to improve the performance of cross - subject brain - activity decoding models by introducing subject embedding, so that they can better generalize to new subjects and provide more abundant insights for neuroscience research.

Group-level Brain Decoding with Deep Learning

Group‐level brain decoding with deep learning

Deep Learning for Micro-Electrocorticographic (µecog) Data

Neural decoding from stereotactic EEG: accounting for electrode variability across subjects

Brain Decoding Using Deep Convolutional Network and Its Application in Cross-Subject Analysis.

Decoding human brain activity with deep learning

Inter-Subject MEG Decoding for Visual Information with Hybrid Gated Recurrent Network

Deep learning of fMRI big data: a novel approach to subject-transfer decoding

MindBridge: A Cross-Subject Brain Decoding Framework

Brain decoding: toward real-time reconstruction of visual perception

Toward Generalizing Visual Brain Decoding to Unseen Subjects

Through their eyes: multi-subject Brain Decoding with simple alignment techniques

Transfer Learning-Based Behavioural Task Decoding from Brain Activity

Deep learning approaches for neural decoding across architectures and recording modalities

The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning

Aligning brain functions boosts the decoding of visual semantics in novel subjects

See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI

Deep Recurrent Encoder: A scalable end-to-end network to model brain signals

Efficient Neural Decoding Based on Multimodal Training

Multi-subject brain decoding with multi-task feature selection.