Abstract:Optical flow captures the motion of pixels in an image sequence over time, providing information about movement, depth, and environmental structure. Flying insects utilize this information to navigate and avoid obstacles, allowing them to execute highly agile maneuvers even in complex environments. Despite its potential, autonomous flying robots have yet to fully leverage this motion information to achieve comparable levels of agility and robustness. Challenges of control from optical flow include extracting accurate optical flow at high speeds, handling noisy estimation, and ensuring robust performance in complex environments. To address these challenges, we propose a novel end-to-end system for quadrotor obstacle avoidance using monocular optical flow. We develop an efficient differentiable simulator coupled with a simplified quadrotor model, allowing our policy to be trained directly through first-order gradient optimization. Additionally, we introduce a central flow attention mechanism and an action-guided active sensing strategy that enhances the policy's focus on task-relevant optical flow observations to enable more responsive decision-making during flight. Our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system demonstrates agile and robust flight in various unknown, cluttered environments in the real world at speeds of up to 6m/s.

Optical Flow Based Obstacle Avoidance and Path Planning for Quadrotor Flight

3D Path Planning of Quadrotor Aerial Robots Using Numerical Optimization

Autonomous Flights in Dynamic Environments with Onboard Vision

Alternating Minimization Based Trajectory Generation for Quadrotor Aggressive Flight

Obstacle Avoidance For Quadrotor Using Improved Method Based On Optical Flow

Integrated Path Planning and Trajectory Tracking Control for Quadrotor UAVs with Obstacle Avoidance in the Presence of Environmental and Systematic Uncertainties: Theory and Experiment

Fusing Stereopsis & Corner Sparse Optical Flow for Real-time Obstacle Avoidance of Unmanned Aerial Vehicle

Seeing Through Pixel Motion: Learning Obstacle Avoidance from Optical Flow with One Camera

Search-based Path Planning and Receding Horizon Based Trajectory Generation for Quadrotor Motion Planning

Visual Autonomous Quadrotor Navigation Using an Improved Artificial Potential Field

Path Planning and Following Control of a Quadrotor Helicopter in Three-Dimensional Space With Limited Information

Perception-Aware Baesd Quadrotor Translation and Yaw Angle Planner in Unpredictable Dynamic Situations

Position estimation and control for quadrotor using optical flow and GPS sensors

Computationally Efficient Trajectory Planning for High Speed Obstacle Avoidance of a Quadrotor with Active Sensing.

Path Planning and Control of a Quadrotor UAV Based on an Improved APF Using Parallel Search

Obstacle-Aware Topological Planning over Polyhedral Representation for Quadrotors.

Nonlinear MPC of Quadrotors for Path Planning and Dynamic Obstacle Avoidance

Quadrotor Trajectory Planning for Visibility-Aware Target Following

Obstacle Avoidance Planning for Quadrotor UAV Based on Improved Adaptive Artificial Potential Field

Aggressive Collision Avoidance and Control Method for Quadrotor

Visual Obstacle Avoidance System Based on Optical Flow Method