Abstract:This paper explores the parallel collaboration of multimodal physiological signals, combining eye tracker output signals, motor imagery, and error-related potentials to control a computer mouse. Specifically, a parallel working mechanism is implemented in the decision layer, where the eye tracker manages cursor movements, and motor imagery manages click functions. Meanwhile, the eye tracker output signals are integrated with electroencephalography data to detect the idle state for asynchronous control. Additionally, error-related potentials evoked by visual feedback, are detected to reduce the cost of error corrections. To efficiently collect data and provide continuous evaluations, we performed offline training and online testing in the designed paradigm. To further validate the practicability, we conducted online experiments on the real-world computer, focusing on a scenario of opening and closing files. The experiments involved seventeen subjects. The results showed that the stability of the eye tracker was optimized from 67.6% to 95.2% by the designed filter, providing the support for parallel control. The accuracy of motor imagery conducted simultaneously with fixations reached 93.41 +/- 2.91%, proving the feasibility of parallel control. Furthermore, the real-world experiments took 45.86 +/- 14.94 s to complete three movements and clicks, and showed a significant improvement compared to the baseline experiment without automatic error correction, validating the practicability of the system and the efficacy of error-related potentials detection. Moreover, this system freed users from the stimulus paradigm, enabling a more natural interaction. To sum up, the parallel collaboration of multimodal physiological signals is novel and feasible, the designed mouse is practical and promising.

A Speed and Direction-Based Cursor Control System with P300 and SSVEP

EEG-based cursor control system

A Hybrid BCI System for 2-D Asynchronous Cursor Control

An EEG-based BCI system for 2-D cursor control by combining Mu/Beta rhythm and P300 potential.

Continuous 2D Control Via State-Machine Triggered by Endogenous Sensory Discrimination and a Fast Brain Switch

Target Selection with Hybrid Feature for BCI-Based 2-D Cursor Control

A high-performance general computer cursor control scheme based on a hybrid BCI combining motor imagery and eye-tracking

Volitional and Real-Time Control Cursor Based on Eye Movement Decoding Using a Linear Decoding Model.

Enhanced P300-Based Cursor Movement Control

The Design of Control System of Cursor Movement Based EEG

A Hybrid BCI System Combining P300 and SSVEP and Its Application to Wheelchair Control.

Study on EEG-based Mouse System by Using Brain-Computer Interface.

SSVEP-Based Brain-Computer Interface Controlled Robotic Platform With Velocity Modulation

Development of a High-speed Mental Spelling System Combining Eye Tracking and SSVEP-based BCI with High Scalability.

Design of Real Time BCI Based on Sensor Technology

Conversion of EEG activity into cursor movement by a brain-computer interface (BCI)

Brain-controlled car system based on SSVEP

Parallel Collaboration and Closed-Loop Control of a Cursor Using Multimodal Physiological Signals

A Usability Study of Low-Cost Wireless Brain-Computer Interface for Cursor Control Using Online Linear Model

Bi-Directional Visual Motion Based Bci Speller

Design And Implementation Of An Asynchronous Bci System With Alpha Rhythm And Ssvep