Abstract:As an alternative to traditional remote controller, research on vision-based hand gesture recognition is being actively conducted in the field of interaction between human and unmanned aerial vehicle (UAV). However, vision-based gesture system has a challenging problem in recognizing the motion of dynamic gesture because it is difficult to estimate the pose of multi-dimensional hand gestures in 2D images. This leads to complex algorithms, including tracking in addition to detection, to recognize dynamic gestures, but they are not suitable for human–UAV interaction (HUI) systems that require safe design with high real-time performance. Therefore, in this paper, we propose a hybrid hand gesture system that combines an inertial measurement unit (IMU)-based motion capture system and a vision-based gesture system to increase real-time performance. First, IMU-based commands and vision-based commands are divided according to whether drone operation commands are continuously input. Second, IMU-based control commands are intuitively mapped to allow the UAV to move in the same direction by utilizing estimated orientation sensed by a thumb-mounted micro-IMU, and vision-based control commands are mapped with hand’s appearance through real-time object detection. The proposed system is verified in a simulation environment through efficiency evaluation with dynamic gestures of the existing vision-based system in addition to usability comparison with traditional joystick controller conducted for applicants with no experience in manipulation. As a result, it proves that it is a safer and more intuitive HUI design with a 0.089 ms processing speed and average lap time that takes about 19 s less than the joystick controller. In other words, it shows that it is viable as an alternative to existing HUI.

A Novel Real-Time Gesture Recognition Algorithm for Human-Robot Interaction on the UAV.

Online Hand Gesture Detection and Recognition for UAV Motion Planning

End-Side Gesture Recognition Method for UAV Control

Real-Time Human-UAV Interaction: New Dataset and Two Novel Gesture-Based Interacting Systems.

Online Control Programming Algorithm for Human–robot Interaction System with a Novel Real-Time Human Gesture Recognition Method

Realtime Human-UAV Interaction Using Deep Learning.

Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding

Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time

Web-based Interactive Drone Control Using Hand Gesture

Multimodal Control of UAV Based on Gesture, Eye Movement and Voice Interaction

Gesture Recognition Based on Localist Attractor Networks with Application to Robot Control [Application Notes]

Gesture-based Piloting of an Aerial Robot using Monocular Vision

Intuitive Human-Robot Interface: A 3-Dimensional Action Recognition and UAV Collaboration Framework

Deep Reinforcement Learning Based Unmanned Aerial Vehicle (UAV) Control Using 3D Hand Gestures

A Novel Gesture Recognition System for Intelligent Interaction with a Nursing-Care Assistant Robot

A Lightweight Gesture Recognition Method on Ultralow-Power Nano-UAVs

A Novel System for Guiding Unmanned Vehicles Based on Human Gesture Recognition.

I Am the UAV: A Wearable Approach for Manipulation of Unmanned Aerial Vehicle

An Intelligent Human–Unmanned Aerial Vehicle Interaction Approach in Real Time Based on Machine Learning Using Wearable Gloves

HGIC: A Hand Gesture Based Interactive Control System for Efficient and Scalable Multi-UAV Operations

UAV Flight Controlling Based on Kinect for Windows V2