Abstract:Artificial intelligence-powered pocket-sized air robots have the potential to revolutionize the Internet-of-Things ecosystem, acting as autonomous, unobtrusive, and ubiquitous smart sensors. With a few cm$^{2}$ form-factor, nano-sized unmanned aerial vehicles (UAVs) are the natural befit for indoor human-drone interaction missions, as the pose estimation task we address in this work. However, this scenario is challenged by the nano-UAVs' limited payload and computational power that severely relegates the onboard brain to the sub-100 mW microcontroller unit-class. Our work stands at the intersection of the novel parallel ultra-low-power (PULP) architectural paradigm and our general development methodology for deep neural network (DNN) visual pipelines, i.e., covering from perception to control. Addressing the DNN model design, from training and dataset augmentation to 8-bit quantization and deployment, we demonstrate how a PULP-based processor, aboard a nano-UAV, is sufficient for the real-time execution (up to 135 frame/s) of our novel DNN, called PULP-Frontnet. We showcase how, scaling our model's memory and computational requirement, we can significantly improve the onboard inference (top energy efficiency of 0.43 mJ/frame) with no compromise in the quality-of-result vs. a resource-unconstrained baseline (i.e., full-precision DNN). Field experiments demonstrate a closed-loop top-notch autonomous navigation capability, with a heavily resource-constrained 27-gram Crazyflie 2.1 nano-quadrotor. Compared against the control performance achieved using an ideal sensing setup, onboard relative pose inference yields excellent drone behavior in terms of median absolute errors, such as positional (onboard: 41 cm, ideal: 26 cm) and angular (onboard: 3.7$^{\circ}$, ideal: 4.1$^{\circ}$).

Robust and Efficient Depth-Based Obstacle Avoidance for Autonomous Miniaturized UAVs

Avoiding dynamic small obstacles with onboard sensing and computating on aerial robots

Autonomous Flights in Dynamic Environments with Onboard Vision

End-to-End Nano-Drone Obstacle Avoidance for Indoor Exploration

Obstacle Avoidance using Monocular Depth Estimation for Small Drone Tello

Stargate: Multimodal Sensor Fusion for Autonomous Navigation On Miniaturized UAVs

Multi-sensory Anti-collision Design for Autonomous Nano-swarm Exploration

Monocular Vision-Based Obstacle Detection/Avoidance For Unmanned Aerial Vehicles

Improving Autonomous Nano-Drones Performance via Automated End-to-End Optimization and Deployment of DNNs

Fusing Stereopsis & Corner Sparse Optical Flow for Real-time Obstacle Avoidance of Unmanned Aerial Vehicle

Bio-inspired Autonomous Exploration Policies with CNN-based Object Detection on Nano-drones

Optimal Multi-Sensor Obstacle Detection System for Small Fixed-Wing UAVs

Onboard dynamic-object detection and tracking for autonomous robot navigation with RGB-D camera

BatDeck: Advancing Nano-drone Navigation with Low-power Ultrasound-based Obstacle Avoidance

An Embedded Platform for Positioning and Obstacle Detection for Small Unmanned Aerial Vehicles

Fully Onboard AI-powered Human-Drone Pose Estimation on Ultra-low Power Autonomous Flying Nano-UAVs

Autonomous Obstacle Avoidance for UAV based on Point Cloud

Quadcopter Drone for Vision-Based Autonomous Target Following

Vision-Less Sensing for Autonomous Micro-Drones

A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera

Robust Vision-based Obstacle Avoidance for Micro Aerial Vehicles in Dynamic Environments