Abstract:Recent studies employ advanced deep convolutional neural networks (CNNs) for monocular depth perception, which can hardly run efficiently on small drones that rely on low/middle-grade GPU(e.g. TX2 and 1050Ti) for computation. In addition, the methods which can effectively and efficiently produce probabilistic depth prediction with a measure of model confidence have not been well studied. The lack of such a method could yield erroneous, sometimes fatal, decisions in drone applications (e.g. selecting a waypoint in a region with a large depth yet a low estimation confidence). This paper presents a real-time onboard approach for monocular depth prediction and obstacle avoidance with a lightweight probabilistic CNN (pCNN), which will be ideal for use in a lightweight energy-efficient drone. For each video frame, our pCNN can efficiently predict its depth map and the corresponding confidence. The accuracy of our lightweight pCNN is greatly boosted by integrating sparse depth estimation from a visual odometry into the network for guiding dense depth and confidence inference. The estimated depth map is transformed into Ego Dynamic Space (EDS) by embedding both dynamic motion constraints of a drone and the confidence values into the spatial depth map. Traversable waypoints are automatically computed in EDS based on which appropriate control inputs for the drone are produced. Extensive experimental results on public datasets demonstrate that our depth prediction method runs at 12Hz and 45Hz on TX2 and 1050Ti GPU respectively, which is 1.8X~5.6X faster than the state-of-the-art methods and achieves better depth estimation accuracy. We also conducted experiments of obstacle avoidance in both simulated and real environments to demonstrate the superiority of our method to the baseline methods.

Monocular Depth Estimation for UAV Obstacle Avoidance

Obstacle Avoidance of Monocular Quadrotors with Depth Estimation

Reactive Obstacle Avoidance of Monocular Quadrotors with Online Adapted Depth Prediction Network.

Autonomous Flights in Dynamic Environments with Onboard Vision

Obstacle Avoidance Methods for Rotor UAVs Using RealSense Camera

Fusing Stereopsis & Corner Sparse Optical Flow for Real-time Obstacle Avoidance of Unmanned Aerial Vehicle

Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotor

A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera

Autonomous quadrotor obstacle avoidance based on dueling double deep recurrent Q-learning with monocular vision

Fast Depth Prediction and Obstacle Avoidance on a Monocular Drone Using Probabilistic Convolutional Neural Network

How Does Monocular Depth Estimation Work for MAV Navigation in the Real World?

Learning the Spatial Perception and Obstacle Avoidance with the Monocular Vision on a Quadrotor

Obstacle Avoidance using Monocular Depth Estimation for Small Drone Tello

Monocular Vision-Based Obstacle Detection/Avoidance For Unmanned Aerial Vehicles

Autonomous Obstacle Avoidance for UAV Based on Fusion of Radar and Monocular Camera

UAV environmental perception and autonomous obstacle avoidance: A deep learning and depth camera combined solution

Obstacle Avoidance for Outdoor Flight of a Quadrotor Based on Computer Vision

Stereo vision based obstacle avoidance strategy for quadcopter UAV

Obstacle visual sensing based on deep learning for low-altitude small unmanned aerial vehicles

Autonomous Obstacle Avoidance for UAV based on Point Cloud

Deep Learning-based Monocular Obstacle Avoidance for Unmanned Aerial Vehicle Navigation in Tree Plantations