Spatio-temporal transformers for decoding neural movement control

Benedetta Candelori,Giampiero Bardella,Indro Spinelli,Surabhi Ramawat,Pierpaolo Pani,Stefano Ferraina,Simone Scardapane

DOI: https://doi.org/10.1101/2024.04.15.589534

2024-10-04

Abstract:Deep learning tools applied to high-resolution neurophysiological data have significantly progressed, offering enhanced decoding, real-time processing, and readability for practical applications. However, the design of artificial neural networks to analyze neural activity remains a challenge, requiring a delicate balance between efficiency in low-data regimes and the interpretability of the results. To this end, we introduce a novel specialized transformer architecture to analyze single-neuron spiking activity. We test our model on multi electrodes recordings from the dorsal premotor cortex (PMd) of non-human primates while performing a motor inhibition task. The proposed architecture provides a very early prediction of the correct movement direction - no later than 230ms after the Go signal presentation across animals - and can accurately forecast whether the movement will be generated or withheld before a Stop signal, unattended, is actually presented. We also analyze the internal dynamics of the model by computing the predicted correlations between time steps and between neurons at successive layers of the architecture. We find that their evolution mirrors previous theoretical analyses. Overall, our framework provides a comprehensive use case for the practical implementation of deep learning tools in motor control research.

Animal Behavior and Cognition

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper attempts to address the problem of efficiently decoding neural activity using deep learning tools, particularly in low-data scenarios. Specifically, the authors propose a new Transformer-based architecture for analyzing the firing activity of individual neurons. The main objectives include:

1. **Predicting movement direction**: By recording neural activity from the dorsal premotor cortex (PMd) using multi-electrode arrays in non-human primates performing a movement inhibition task, the model can predict the correct movement direction in advance, completing the prediction within 230 milliseconds after the "Go" signal is presented at the latest.

2. **Predicting movement generation and inhibition**: The model can accurately predict whether a movement will be generated or inhibited before the actual "Stop" signal is presented.

3. **Improving model interpretability**: By calculating the predictive correlation between different time steps and neurons, analyzing the internal dynamics of the model, validating model performance, and providing interpretable attention maps, which can be compared with experimental results and theoretical models.

4. **Addressing data analysis challenges**: One of the main challenges in analyzing neural activity data is maintaining model efficiency and result interpretability in low-data scenarios. This study aims to balance these needs by designing a specialized Transformer architecture.

In summary, this paper introduces a novel spatiotemporal Transformer architecture to provide a comprehensive solution for decoding neural activity in motor control research, enhancing both prediction accuracy and model interpretability and practicality.