Abstract:Optical flow captures the motion of pixels in an image sequence over time, providing information about movement, depth, and environmental structure. Flying insects utilize this information to navigate and avoid obstacles, allowing them to execute highly agile maneuvers even in complex environments. Despite its potential, autonomous flying robots have yet to fully leverage this motion information to achieve comparable levels of agility and robustness. Challenges of control from optical flow include extracting accurate optical flow at high speeds, handling noisy estimation, and ensuring robust performance in complex environments. To address these challenges, we propose a novel end-to-end system for quadrotor obstacle avoidance using monocular optical flow. We develop an efficient differentiable simulator coupled with a simplified quadrotor model, allowing our policy to be trained directly through first-order gradient optimization. Additionally, we introduce a central flow attention mechanism and an action-guided active sensing strategy that enhances the policy's focus on task-relevant optical flow observations to enable more responsive decision-making during flight. Our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system demonstrates agile and robust flight in various unknown, cluttered environments in the real world at speeds of up to 6m/s.

Obstacle Avoidance of Monocular Quadrotors with Depth Estimation

Autonomous Flights in Dynamic Environments with Onboard Vision

Monocular Vision Based Obstacle Detection for Robot Navigation in Unstructured Environment

Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotor

Fusing Stereopsis & Corner Sparse Optical Flow for Real-time Obstacle Avoidance of Unmanned Aerial Vehicle

Autonomous quadrotor obstacle avoidance based on dueling double deep recurrent Q-learning with monocular vision

A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera

Real-time Quadrotor Navigation Through Planning in Depth Space in Unstructured Environments

Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning

Monocular Vision-Based Obstacle Detection/Avoidance For Unmanned Aerial Vehicles

Stereo vision based obstacle avoidance strategy for quadcopter UAV

Obstacle Avoidance using Monocular Depth Estimation for Small Drone Tello

Seeing Through Pixel Motion: Learning Obstacle Avoidance from Optical Flow with One Camera

3D ground plane estimation from a monocular vehicle-borne image

3-D Reciprocal Collision Avoidance on Physical Quadrotor Helicopters with On-Board Sensing for Relative Positioning

Quadcopter Drone for Vision-Based Autonomous Target Following

Error modelling of depth estimation based on simplified stereo vision for mobile robots

Avoiding dynamic small obstacles with onboard sensing and computating on aerial robots

Obstacle Avoidance Using Stereo Camera

Monocular Camera-based Complex Obstacle Avoidance via Efficient Deep Reinforcement Learning

Visual path following with obstacle avoidance for quadcopters in indoor environments