Towards a "universal translator" for neural dynamics at single-cell, single-spike resolution

Yizi Zhang,Yanchen Wang,Donato Jimenez-Beneto,Zixuan Wang,Mehdi Azabou,Blake Richards,Olivier Winter,International Brain Laboratory,Eva Dyer,Liam Paninski,Cole Hurwitz

2024-07-20

Abstract:Neuroscience research has made immense progress over the last decade, but our understanding of the brain remains fragmented and piecemeal: the dream of probing an arbitrary brain region and automatically reading out the information encoded in its neural activity remains out of reach. In this work, we build towards a first foundation model for neural spiking data that can solve a diverse set of tasks across multiple brain areas. We introduce a novel self-supervised modeling approach for population activity in which the model alternates between masking out and reconstructing neural activity across different time steps, neurons, and brain regions. To evaluate our approach, we design unsupervised and supervised prediction tasks using the International Brain Laboratory repeated site dataset, which is comprised of Neuropixels recordings targeting the same brain locations across 48 animals and experimental sessions. The prediction tasks include single-neuron and region-level activity prediction, forward prediction, and behavior decoding. We demonstrate that our multi-task-masking (MtM) approach significantly improves the performance of current state-of-the-art population models and enables multi-task learning. We also show that by training on multiple animals, we can improve the generalization ability of the model to unseen animals, paving the way for a foundation model of the brain at single-cell, single-spike resolution.

Neurons and Cognition,Machine Learning,Neural and Evolutionary Computing

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the fragmentation and limitations in the understanding of the brain in current neuroscience research. Specifically, although neuroscience research has made great progress in the past decade, the understanding of the brain is still limited to small - scale neural circuits in specific brain regions, and these studies are usually carried out in a standardized behavioral context. This has led to the fact that the existing neural population models are often specific to brain regions and are designed for specific experimental situations, which limits their effectiveness in broader applications and their insight into distributed brain functions.

To address these challenges, the authors propose a Multi - task - Masking (MtM) method, aiming to construct a basic model that can process data from multiple brain regions and can solve a variety of prediction tasks, such as single - neuron activity prediction, regional - level activity prediction, forward prediction, and behavior decoding. Through this method, the authors hope to seamlessly "translate" neural activity at different scales (including the population level, regional level, and single - neuron level), thus providing a new tool for understanding and simulating brain functions.

In addition, the paper also explores how to improve the generalization ability of the model for unseen animals through data training across multiple animals, and how to use prompt technology to adapt to different downstream tasks at the test time, thereby further enhancing the performance and application scope of the model.