Abstract:Brain-computer interfaces (BCI) using electroencephalography (EEG) provide a non-invasive method for users to interact with external devices without the need for muscle activation. While noninvasive BCIs have the potential to improve the quality of lives of healthy and motor impaired individuals, they currently have limited applications due to inconsistent performance and low degrees of freedom. In this study, we use deep learning (DL)-based decoders for online Continuous Pursuit (CP), a complex BCI task requiring the user to track an object in two-dimensional space. We developed a labeling system to use CP data for supervised learning, trained DL-based decoders based on two architectures, including a newly proposed adaptation of the PointNet architecture, and evaluated the performance over several online sessions. We rigorously evaluated the DL-based decoders in a total of 28 human participants, and found that the DL-based models improved throughout the sessions as more training data became available and significantly outperformed a traditional BCI decoder by the last session. We also performed additional experiments to test an implementation of transfer learning by pre-training models on data from other subjects, and mid-session training to reduce inter-session variability. The results from these experiments showed that pre-training did not significantly improve performance, but updating the models mid-session may have some benefit. Overall, these findings support the use of DL-based decoders for improving BCI performance in complex tasks like CP, which can expand the potential applications of BCI devices and help improve the quality of lives of healthy and motor-impaired individuals.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to improve the performance of non-invasive Brain-Computer Interface (BCI) systems in Continuous Pursuit (CP) tasks through a deep learning (DL)-based decoder. Specifically, the paper addresses the following key issues: 1. **Enhancing BCI Performance**: - Current non-invasive BCI systems have limited application due to inconsistent performance and low degrees of freedom. By introducing deep learning technology, the paper attempts to enhance the performance of BCI systems, making them more applicable to real-world scenarios. 2. **Continuous Control in Complex Tasks**: - Traditional BCI tasks typically use a Discrete Trial (DT) paradigm, where users only need to move a cursor to a fixed target position in each trial. In contrast, CP tasks require users to continuously track a randomly moving target in a 2D space, necessitating dynamic and continuous control. The paper develops new labeling systems and trains DL models to enable BCI systems to handle such complex continuous tasks. 3. **Reducing Training Data Requirements**: - Deep learning models usually require a large amount of training data to achieve good performance. The paper explores Transfer Learning (TL) and Recalibration (CL) methods to reduce the dependency on large training datasets and improve the model's adaptability across different sessions. 4. **Improving Real-Time Performance**: - In CP tasks, BCI decoders may be affected by latency, i.e., the time difference between the change in target direction and the decoder update. The paper analyzes the performance of different decoders under various latencies and explores ways to mitigate the impact of latency on performance. ### Main Findings - **Performance Improvement of DL Models**: - The study finds that with an increase in training data, DL-based decoders (such as EEGNet and PointNet) gradually outperform traditional decoders across multiple metrics, including cursor-target distance, correlation, and mean deviation angle. - **Effectiveness of Transfer Learning**: - Transfer learning does not significantly improve performance in early sessions, but recalibration methods help enhance performance in certain cases. - **Optimization of Real-Time Performance**: - Adjusting latency can significantly improve the real-time performance of DL models in CP tasks. ### Significance These findings support the use of DL-based decoders to improve BCI system performance in complex tasks, thereby expanding the application range of BCI devices and enhancing the daily life quality of healthy individuals and patients with motor impairments. Specifically, through continuous pursuit tasks, BCI systems can achieve continuous control of devices such as robots, providing important technical support for future practical applications.

Continuous Tracking using Deep Learning-based Decoding for Non-invasive Brain-Computer Interface

Continuous tracking using deep learning-based decoding for noninvasive brain-computer interface

Extracting Semantic-Dynamic Features for Long-Term Stable Brain Computer Interface.

Tracking Neural Modulation Depth by Dual Sequential Monte Carlo Estimation on Point Processes for Brain–Machine Interfaces

Meeting brain–computer interface user performance expectations using a deep neural network decoding framework

A Human-Machine Joint Learning Framework to Boost Endogenous BCI Training

Real-time Decoding Algorithm in Brain Machine Interfaces

Design a Novel BCI for Neurorehabilitation Using Concurrent LFP and EEG Features: A Case Study

Deep Learning Based Inter-subject Continuous Decoding of Motor Imagery for Practical Brain-Computer Interfaces

Quantized Attention-Gated Kernel Reinforcement Learning for Brain-Machine Interface Decoding.

Deep Learning Decoding of Mental State in Non-invasive Brain Computer Interface

Kernel Reinforcement Learning-Assisted Adaptive Decoder Facilitates Stable and Continuous Brain Control Tasks

Translating Mental Imaginations into Characters with Codebooks and Dynamics-Enhanced Decoding

Visual tracking brain computer interface

Non-invasive Decoding of Hand Movements from Electroencephalography Based on a Hierarchical Linear Regression Model.

Continuous and discrete decoding of overt speech with electroencephalography

Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG)

A Semi-Supervised Progressive Learning Algorithm for Brain–Computer Interface

Geometric neural network based on phase space for BCI-EEG decoding

Personalized Continual EEG Decoding Framework for Knowledge Retention and Transfer

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain-Computer Interfaces