Abstract:Objective. The electroencephalography (EEG)-based brain–computer interfaces (BCIs) have been used in the control of robotic arms. The performance of non-invasive BCIs may not be satisfactory due to the poor quality of EEG signals, so the shared control strategies were tried as an alternative solution. However, most of the existing shared control methods set the arbitration rules manually, which highly depended on the specific tasks and developer's experience. In this study, we proposed a novel shared control model that automatically optimized the control commands in a dynamical way based on the context in real-time control. Besides, we employed the hybrid BCI to better allocate commands with multiple functions. The system allowed non-invasive BCI users to manipulate a robotic arm moving in a three-dimensional (3D) space and complete a pick-place task of multiple objects. Approach. Taking the scene information obtained by computer vision as a knowledge base, a machine agent was designed to infer the user's intention and generate automatic commands. Based on the inference confidence and user's characteristic, the proposed shared control model fused the machine autonomy and human intention dynamically for robotic arm motion optimization during the online control. In addition, we introduced a hybrid BCI scheme that applied steady-state visual evoked potentials and motor imagery to the divided primary and secondary BCI interfaces to better allocate the BCI resources (e.g. decoding computing power, screen occupation) and realize the multi-dimensional control of the robotic arm. Main results. Eleven subjects participated in the online experiments of picking and placing five objects that scattered at different positions in a 3D workspace. The results showed that most of the subjects could control the robotic arm to complete accurate and robust picking task with an average success rate of approximately 85% under the shared control strategy, while the average success rate of placing task controlled by pure BCI was 50% approximately. Significance. In this paper, we proposed a novel shared controller for motion automatic optimization, together with a hybrid BCI control scheme that allocated paradigms according to the importance of commands to realize multi-dimensional and effective control of a robotic arm. Our study indicated that the shared control strategy with hybrid BCI could greatly improve the performance of the brain-actuated robotic arm system.

Noninvasive Brain-computer Interface Based High-level Control of a Robotic Arm for Pick and Place Tasks

Combination of high-frequency SSVEP-based BCI and computer vision for controlling a robotic arm

Control of a 7-DOF Robotic Arm System with an SSVEP-Based BCI.

Combination of Augmented Reality Based Brain- Computer Interface and Computer Vision for High-Level Control of a Robotic Arm.

Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks

Control of The Robotic Arm System With an SSVEP-Based BCI

Continuous 2D Control Via State-Machine Triggered by Endogenous Sensory Discrimination and a Fast Brain Switch

BCI Control of a Robotic Arm Based on SSVEP With Moving Stimuli for Reach and Grasp Tasks

A brain-actuated robotic arm system using non-invasive hybrid brain–computer interface and shared control strategy

Continuous Hybrid BCI Control for Robotic Arm Using Noninvasive Electroencephalogram, Computer Vision, and Eye Tracking

A Hybrid BCI Based on SSVEP and EOG for Robotic Arm Control

Author Correction: Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks

Adaptive asynchronous control system of robotic arm based on augmented reality-assisted brain–computer interface

Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance

A SSVEP-Based BCI for Controlling a 4-DOF Robotic Manipulator

A Robotic Arm Control System with Simultaneous and Sequential Modes Combining Eye-Tracking with Steady-State Visual Evoked Potential in Virtual Reality Environment.

Robotic Precise Grasping with Varied Grasp Patterns Utilizing Brain-Computer Interface and Computer Vision

Control of a Robotic Arm With an Optimized Common Template-Based CCA Method for SSVEP-Based BCI

Implementation of a Brain-Computer Interface for Robotic Arm Control

SSVEP-Based Brain-Computer Interface Controlled Robotic Platform With Velocity Modulation

An EEG-/EOG-Based Hybrid Brain-Computer Interface: Application on Controlling an Integrated Wheelchair Robotic Arm System