Neural decoding from stereotactic EEG: accounting for electrode variability across subjects

Georgios Mentzelopoulos,Evangelos Chatzipantazis,Ashwin G. Ramayya,Michelle J. Hedlund,Vivek P. Buch,Kostas Daniilidis,Konrad P. Kording,Flavia Vitale

2024-11-02

Abstract:Deep learning based neural decoding from stereotactic electroencephalography (sEEG) would likely benefit from scaling up both dataset and model size. To achieve this, combining data across multiple subjects is crucial. However, in sEEG cohorts, each subject has a variable number of electrodes placed at distinct locations in their brain, solely based on clinical needs. Such heterogeneity in electrode number/placement poses a significant challenge for data integration, since there is no clear correspondence of the neural activity recorded at distinct sites between individuals. Here we introduce seegnificant: a training framework and architecture that can be used to decode behavior across subjects using sEEG data. We tokenize the neural activity within electrodes using convolutions and extract long-term temporal dependencies between tokens using self-attention in the time dimension. The 3D location of each electrode is then mixed with the tokens, followed by another self-attention in the electrode dimension to extract effective spatiotemporal neural representations. Subject-specific heads are then used for downstream decoding tasks. Using this approach, we construct a multi-subject model trained on the combined data from 21 subjects performing a behavioral task. We demonstrate that our model is able to decode the trial-wise response time of the subjects during the behavioral task solely from neural data. We also show that the neural representations learned by pretraining our model across individuals can be transferred in a few-shot manner to new subjects. This work introduces a scalable approach towards sEEG data integration for multi-subject model training, paving the way for cross-subject generalization for sEEG decoding.

Signal Processing,Machine Learning,Neurons and Cognition

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is how to overcome the challenge of electrode number and location heterogeneity among different subjects when performing neural decoding using stereoelectroencephalography (sEEG). Specifically, because the number and location of electrodes for each subject vary according to clinical needs, this leads to significant difficulties in cross - subject data integration. This heterogeneity makes it very difficult to find the correspondence between the neural activities recorded among different subjects, thereby limiting the effective training and generalization ability of multi - subject models.

To solve this problem, the paper introduces a training framework and architecture named seegnificant, aiming to achieve cross - subject neural decoding in the following ways:

1. **Convolutional Tokenization**: Tokenize the neural activity within each electrode and use convolutional operations to extract features in the time dimension.
2. **Self - Attention Mechanism**: First, apply the self - attention mechanism in the time dimension to capture long - term time - dependencies; then encode the position information of each electrode (through its MNI coordinates) into the features and apply the self - attention mechanism again in the electrode dimension to capture long - distance spatial - dependencies.
3. **Personalized Task Heads**: Use subject - specific task heads to adapt to individual differences, and these task heads can be used for downstream decoding tasks.

Through this method, the paper constructs a multi - subject model that can be trained from the combined data of 21 subjects and successfully decodes the trial reaction time of subjects in behavioral tasks. In addition, the study also shows that through the pre - trained model, the learned neural representations can be effectively transferred to new subjects with a small number of training samples, further verifying the generalization ability and practicality of this method.