Abstract:In autonomous and mobile robotics, a principal challenge is resilient real-time environmental perception, particularly in situations characterized by unknown and dynamic elements, as exemplified in the context of autonomous drone racing. This study introduces a perception technique for detecting drone racing gates under illumination variations, which is common during high-speed drone flights. The proposed technique relies upon a lightweight neural network backbone augmented with capabilities for continual learning. The envisaged approach amalgamates predictions of the gates' positional coordinates, distance, and orientation, encapsulating them into a cohesive pose tuple. A comprehensive number of tests serve to underscore the efficacy of this approach in confronting diverse and challenging scenarios, specifically those involving variable lighting conditions. The proposed methodology exhibits notable robustness in the face of illumination variations, thereby substantiating its effectiveness.

What problem does this paper attempt to address?

This paper focuses on how to achieve reliable gate detection in the context of autonomous drone racing under dynamic illumination conditions, which is a crucial environmental perception challenge. The study proposes a method based on lightweight neural networks and incorporates the capability of continual learning to adapt to changes in illumination. This method predicts the position, distance, and direction of the gate, combining them into a complete pose tuple. The experiments demonstrate that this approach exhibits significant robustness in different illumination conditions and effectively addresses the impact of illumination variations. In early work on drone racing, the gate perception systems relied on manually designed methods, but these approaches may not be robust enough under real-world conditions. With the development of deep learning, researchers have begun to utilize deep neural networks to improve the handling of uncertainty in perception tasks. Nevertheless, traditional models still face difficulties in generalizing among different lighting scenes. Continual learning can address this issue by allowing models to adapt to changing environments without forgetting previous knowledge, thereby enhancing detection performance under illumination variations. The paper introduces a gate detection framework based on continual learning. Unlike previous methods, this framework does not require storing all training samples but only trains on samples specific to the given task. Experimental results show that this method achieves stable gate detection under dynamic illumination conditions, validating its effectiveness and robustness. The experimental section includes incremental learning under two illumination conditions: from bright to dark and from dark to bright. It demonstrates the impact of different continual learning strategies, such as knowledge distillation and replay, on performance. The results indicate that replay strategies, particularly Dark Experience Replay (DER++) combined with knowledge distillation, are most effective in mitigating catastrophic forgetting and improving overall performance. Additionally, the learning order (from simple to complex) also significantly affects the effectiveness of continual learning.

Continual Learning for Robust Gate Detection under Dynamic Lighting in Autonomous Drone Racing

Autonomous Flights in Dynamic Environments with Onboard Vision

Real-Time Drone Detection Using Deep Learning Approach.

PencilNet: Zero-Shot Sim-to-Real Transfer Learning for Robust Gate Perception in Autonomous Drone Racing

A Training-time Friendly Network for Real-time Drone Detection.

Image Generation for Efficient Neural Network Training in Autonomous Drone Racing

Flying through Moving Gates without Full State Estimation

Deep Drone Racing: Learning Agile Flight in Dynamic Environments

Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing

Vision in adverse weather: Augmentation using CycleGANs with various object detectors for robust perception in autonomous racing

Demonstrating Agile Flight from Pixels without State Estimation

Autonomous Drone Racing: A Survey

Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions

Visual Model-predictive Localization for Computationally Efficient Autonomous Racing of a 72-gram Drone

Learning Agile Flight Maneuvers: Deep SE(3) Motion Planning and Control for Quadrotors

An integrated solution for an autonomous drone racing in indoor environments

Vision Based Drone Obstacle Avoidance by Deep Reinforcement Learning

DeepPilot: A CNN for Autonomous Drone Racing

Gate-Aware Online Planning for Two-Player Autonomous Drone Racing

A Sim-to-Real Deep Learning-Based Framework for Autonomous Nano-Drone Racing