Abstract:ObjectiveThe difficulty of unmanned aerial vehicle (UAV) control recently lies in multidirectional movement in 3-dimensional space, improving control accuracy and manipulation safety. To address these challenges, a UAV control system that incorporates a hybrid brain-computer interface (hBCI), gyroscope and visual obstacle avoidance based on monocular depth estimation is proposed. Approach. We propose an efficient steady-state visual evoked potential (SSVEP) classification network (CL-NET) featuring a one-dimensional convolutional neural network, a long short-term memory module and an attention module to identify the user's intention for UAV movement in the front, back, left and right directions. The take-off, landing and rising control of the UAV is realized by an electrooculogram (EOG) signal detection algorithm, a blink state detector. In addition, the UAV can fly in an oblique state and rotate according to the current head posture detected by a gyroscope. Furthermore, an improved monocular depth estimation network is employed to design the autonomous obstacle avoidance module of the UAV, ensuring the safety of the brain-controlled system in practice. Main results. The proposed CL-NET delivers an accuracy of 98.67% on the public dataset and an accuracy of 97.92% on the self-collected dataset, both of which surpass the performance of state-of-the-art models. Additionally, we set up a brain control group and a remote control group to conduct practical experiments in a realistic environment. In the experiments involving sixteen subjects, the proposed UAV control system reached an average information transfer rate (ITR) of 44.09 bits/min, and the brain control group had a lower collision rate than the remote control group. Significance. The hybrid control method ensures that the multi-degree-of-freedom (multi-DOF) UAV control system maintains outstanding performance while ensuring good safety.

Research on Multi-modal Interactive Control for Quadrotor UAV

Multimodal Control of UAV Based on Gesture, Eye Movement and Voice Interaction

Web-based Interactive Drone Control Using Hand Gesture

Research on the UAV Multi-Channel Human-Machine Interaction System

Natural Interactive Control Method for Unmanned Swarm Systems Based on Gesture Recognition

Multi-degree-of-freedom Unmanned Aerial Vehicle Control Combining a Hybrid Brain-Computer Interface and Visual Obstacle Avoidance

Intelligent command and control of UAV Based on brain computer and eye tracking combined technology

Control of a Quadrotor on a Mobile Device Using Machine Learning-Based Monocular Gaze Tracking

A Multiple Command UAV Control System Based on a Hybrid Brain-Computer Interface.

Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time

A Novel Real-Time Gesture Recognition Algorithm for Human-Robot Interaction on the UAV.

Control System Design and Implementation of a Quadrotor Unmanned Aerial Vehicle

UAV Flight Controlling Based on Kinect for Windows V2

Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding

HGIC: A Hand Gesture Based Interactive Control System for Efficient and Scalable Multi-UAV Operations

A VR-based BCI Interactive System for UAV Swarm Control

Real-Time Human-UAV Interaction: New Dataset and Two Novel Gesture-Based Interacting Systems.

End-Side Gesture Recognition Method for UAV Control

An Intuitive End-to-End Human-UAV Interaction System for Field Exploration.

Research on Aerial Autonomous Docking and Landing Technology of Dual Multi-Rotor UAV

Motion Control of a Hybrid Quadruped-Quadrotor Robot