Abstract:Autonomous Drone Racing (ADR) was first proposed in IROS 2016. It called for the development of an autonomous drone capable of beating a human in a drone race. After almost five years, several teams have proposed different solutions with a common pipeline: gate detection; drone localization; and stable flight control. Recently, Deep Learning (DL) has been used for gate detection and localization of the drone regarding the gate. However, recent competitions such as the Game of Drones, held at NeurIPS 2019, called for solutions where DL played a more significant role. Motivated by the latter, in this work, we propose a CNN approach called DeepPilot that takes camera images as input and predicts flight commands as output. These flight commands represent: the angular position of the drone's body frame in the roll and pitch angles, thus producing translation motion in those angles; rotational speed in the yaw angle; and vertical speed referred as altitude h. Values for these 4 flight commands, predicted by DeepPilot, are passed to the drone's inner controller, thus enabling the drone to navigate autonomously through the gates in the racetrack. For this, we assume that the next gate becomes visible immediately after the current gate has been crossed. We present evaluations in simulated racetrack environments where DeepPilot is run several times successfully to prove repeatability. In average, DeepPilot runs at 25 frames per second (fps). We also present a thorough evaluation of what we called a temporal approach, which consists of creating a mosaic image, with consecutive camera frames, that is passed as input to the DeepPilot. We argue that this helps to learn the drone's motion trend regarding the gate, thus acting as a local memory that leverages the prediction of the flight commands. Our results indicate that this purely DL-based artificial pilot is feasible to be used for the ADR challenge.

Champion-level drone racing using deep reinforcement learning

A Trajectory-based Flight Assistive System for Novice Pilots in Drone Racing Scenario

Autonomous Multi-Drone Racing Method Based on Deep Reinforcement Learning

Autonomous Drone Racing with Deep Reinforcement Learning

Deep Drone Racing: Learning Agile Flight in Dynamic Environments

A Sim-to-Real Deep Learning-based Framework for Autonomous Nano-drone Racing

Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing

AlphaPilot: autonomous drone racing

The Artificial Intelligence behind the winning entry to the 2019 AI Robotic Racing Competition

Autonomous Single-Image Drone Exploration With Deep Reinforcement Learning and Mixed Reality

Autonomous Drone Racing: A Survey

DeepPilot: A CNN for Autonomous Drone Racing

Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning

Demonstrating Agile Flight from Pixels without State Estimation

Visual Model-predictive Localization for Computationally Efficient Autonomous Racing of a 72-gram Drone

DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning

Long-Term Planning with Deep Reinforcement Learning on Autonomous Drones

Vision Based Drone Obstacle Avoidance by Deep Reinforcement Learning

FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing