Abstract:Long-term, stable, and real-time decoding of behavior-dependent neural dynamics from the same cells is critical for brain-computer interfaces (BCIs) and for understanding neural evolution during learning, and memory, and disease progression. Recent advances in flexible and high-density electrodes have enabled the stability required for long-term tracking but generate vast datasets that challenge existing analysis methods. Current spike sorting approaches rely heavily on manual curation and lack scalability for large-scale, real-time processing. Here, we introduce AutoSort, an end-to-end multimodal deep neural network-based method that enables real-time tracking and decoding of the same neurons over months. AutoSort uses a scalable strategy by learning deep representations from initial recordings and applying the trained model in real-time. It integrates multimodal features, including waveform features, distribution patterns, and inferred neuron spatial locations, to ensure robustness and accuracy. AutoSort outperforms existing methods in both simulated and long-term recordings, reducing computational demands by using only 10% of the time and 25% of the memory compared to conventional methods. By combining AutoSort with high-density flexible probes, we track neural dynamics in real-time during motor learning and skill acquisition over 2 months, capturing intrinsic neural manifold drift, stabilization, and post-learning representational drift. AutoSort offers a promising solution for studying long-term neural intrinsic dynamics and enabling real-time BCI decoding.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to achieve long - term (lasting for several months), stable, and real - time behavior - dependent neural dynamics decoding on the same group of neurons. Specifically, the research aims to overcome the challenges encountered by existing methods when dealing with large - scale, long - time neural recording data, such as high requirements for manual proofreading and difficulty in expanding to large - scale real - time processing. ### Main problem decomposition: 1. **Long - term and stable neural activity decoding**: - **Background**: Brain - computer interfaces (BCIs) and understanding neural evolution in learning, memory, and disease progression require long - term and stable decoding of behavior - dependent neural dynamics from the same cells. - **Challenge**: Traditional electrodes (such as rigid and bulky probes) cannot maintain stable recordings because probe drift and immune responses lead to mechanical mismatches, which affect the stable decoding of neural activities. 2. **Big data processing and real - time decoding**: - **Background**: Although the development of flexible electrodes and high - density microelectrode arrays has provided stability, it has generated huge data sets, posing great challenges to existing analysis methods. - **Challenge**: Existing spike sorting methods rely on manual proofreading, lack scalability, and cannot achieve large - scale real - time processing. 3. **Automatic alignment and consistency of neural signals**: - **Background**: In order to achieve long - term and stable brain - computer interface decoding, it is necessary to ensure the consistency and accuracy of neural signals on a time scale from days to months. - **Challenge**: Current methods have difficulty in maintaining a consistent neuron count and have an exponentially increasing computational demand in long - time recordings. ### Solution: To solve the above problems, the authors propose AutoSort, an end - to - end multimodal deep - learning method for real - time tracking and decoding of the activity of the same group of neurons for up to several months. AutoSort solves these problems in the following ways: - **Multimodal feature integration**: Combine waveform features, distribution patterns, and inferred neuron spatial positions to ensure robustness and accuracy. - **Efficient real - time processing**: Through deep - learning model training, only the initial recording requires manual proofreading, and subsequent processing is fully automated, greatly reducing time and memory consumption. - **Long - term stable tracking**: Be able to maintain consistent neuron tracking in long - time recordings, and capture neural manifold drift, stabilization, and post - learning representation drift. Through these innovations, AutoSort provides a promising solution for studying long - term neural intrinsic dynamics and achieving real - time brain - computer interface decoding.

End-to-end multimodal deep learning for real-time decoding of months-long neural activity from the same cells

Extracting Semantic-Dynamic Features for Long-Term Stable Brain Computer Interface.

A Hybrid Autoencoder Framework of Dimensionality Reduction for Brain-Computer Interface Decoding

Decoding the Nonstationary Neural Activity in Motor Cortex for Brain Machine Interfaces

Local-learning-based Neuron Selection for Grasping Gesture Prediction in Motor Brain Machine Interfaces

Spike Detection and Spike Sorting with a Hidden Markov Model Improves Offline Decoding of Motor Cortical Recordings

NeuSort: An Automatic Adaptive Spike Sorting Approach with Neuromorphic Models

A neural network for online spike classification that improves decoding accuracy

Unsupervised decoding of long-term, naturalistic human neural recordings with automated video and audio annotations

Adaptive SpikeDeep-Classifier: Self-organizing and self-supervised machine learning algorithm for online spike sorting

Towards a "universal translator" for neural dynamics at single-cell, single-spike resolution

Continuous tracking using deep learning-based decoding for noninvasive brain-computer interface

Dynamic Ensemble Modeling Approach to Nonstationary Neural Decoding in Brain-Computer Interfaces

Recording and Analyzing Multimodal Large-Scale Neuronal Ensemble Dynamics on CMOS-Integrated High-Density Microelectrode Array

Multiscale fusion enhanced spiking neural network for invasive BCI neural signal decoding

An Accurate and Robust Method for Spike Sorting Based on Convolutional Neural Networks.

Continuous Tracking using Deep Learning-based Decoding for Non-invasive Brain-Computer Interface

Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning

Compact Standalone Platform for Neural Recording with Real-Time Spike Sorting and Data Logging

Deep learning approaches for neural decoding across architectures and recording modalities

An Auto-Segmented Multi-Time Window Dual-Scale Neural Network for Brain-Computer Interfaces Based on Event-Related Potentials