Abstract:Objective. The electroencephalography (EEG)-based brain–computer interfaces (BCIs) have been used in the control of robotic arms. The performance of non-invasive BCIs may not be satisfactory due to the poor quality of EEG signals, so the shared control strategies were tried as an alternative solution. However, most of the existing shared control methods set the arbitration rules manually, which highly depended on the specific tasks and developer's experience. In this study, we proposed a novel shared control model that automatically optimized the control commands in a dynamical way based on the context in real-time control. Besides, we employed the hybrid BCI to better allocate commands with multiple functions. The system allowed non-invasive BCI users to manipulate a robotic arm moving in a three-dimensional (3D) space and complete a pick-place task of multiple objects. Approach. Taking the scene information obtained by computer vision as a knowledge base, a machine agent was designed to infer the user's intention and generate automatic commands. Based on the inference confidence and user's characteristic, the proposed shared control model fused the machine autonomy and human intention dynamically for robotic arm motion optimization during the online control. In addition, we introduced a hybrid BCI scheme that applied steady-state visual evoked potentials and motor imagery to the divided primary and secondary BCI interfaces to better allocate the BCI resources (e.g. decoding computing power, screen occupation) and realize the multi-dimensional control of the robotic arm. Main results. Eleven subjects participated in the online experiments of picking and placing five objects that scattered at different positions in a 3D workspace. The results showed that most of the subjects could control the robotic arm to complete accurate and robust picking task with an average success rate of approximately 85% under the shared control strategy, while the average success rate of placing task controlled by pure BCI was 50% approximately. Significance. In this paper, we proposed a novel shared controller for motion automatic optimization, together with a hybrid BCI control scheme that allocated paradigms according to the importance of commands to realize multi-dimensional and effective control of a robotic arm. Our study indicated that the shared control strategy with hybrid BCI could greatly improve the performance of the brain-actuated robotic arm system.

Embodied Intelligence: Bionic Robot Controller Integrating Environment Perception, Autonomous Planning, and Motion Control

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Dexterous Manipulation Control of a Bionic Prosthesis in Cooperation with Human Upper Limbs

Perception Enhanced Robotic Arm

“Follower” to “Collaborator”: A robot proactive collaborative controller based on human multimodal information for 3D handling/assembly scenarios

Vision-Based Robotic Manipulation of Intelligent Wheelchair with Human-Computer Shared Control

Design Of An Ssvep-Based Bci System With Vision Assisted Navigation Module For The Cooperative Control Of Multiple Robots

Unifying 3D Representation and Control of Diverse Robots with a Single Camera

Design of intelligent 3D collaborative manufacturing platform for non-holonomic mobile industrial robots based on improved binocular vision

Multi-scale Control and Action Recognition Based Human-Robot Collaboration Framework Facing New Generation Intelligent Manufacturing

Semi-Autonomous Robotic Arm Reaching With Hybrid Gaze-Brain Machine Interface

Motion Control Method of Bionic Robot Dog Based on Vision and Navigation Information

Proactive Robot Control for Collaborative Manipulation Using Human Intent

Reconfigurable wireless control system for a dual-arm cooperative robotic system

A Control Scheme for Smooth Transition in Physical Human-Robot-Environment Between Two Modes: Augmentation and Autonomous

Bridging Intelligence and Instinct: A New Control Paradigm for Autonomous Robots

A Rgb-D Sensor And Speech Interaction Based Object Manipulation System With Improved Cbr-Bdi Reasoning

Eclectic Theory of Intelligent Robots

A brain-actuated robotic arm system using non-invasive hybrid brain–computer interface and shared control strategy

Brain–Computer Interface and Hand-Guiding Control in a Human–Robot Collaborative Assembly Task

Development of Sensory-Motor Fusion-Based Manipulation and Grasping Control for a Robotic Hand-Eye System