Abstract:The rapid advancement of energy-efficient parallel ultra-low-power (ULP) ucontrollers units (MCUs) is enabling the development of autonomous nano-sized unmanned aerial vehicles (nano-UAVs). These sub-10cm drones represent the next generation of unobtrusive robotic helpers and ubiquitous smart sensors. However, nano-UAVs face significant power and payload constraints while requiring advanced computing capabilities akin to standard drones, including real-time Machine Learning (ML) performance and the safe co-existence of general-purpose and real-time OSs. Although some advanced parallel ULP MCUs offer the necessary ML computing capabilities within the prescribed power limits, they rely on small main memories (<1MB) and ucontroller-class CPUs with no virtualization or security features, and hence only support simple bare-metal runtimes. In this work, we present Shaheen, a 9mm2 200mW SoC implemented in 22nm FDX technology. Differently from state-of-the-art MCUs, Shaheen integrates a Linux-capable RV64 core, compliant with the v1.0 ratified Hypervisor extension and equipped with timing channel protection, along with a low-cost and low-power memory controller exposing up to 512MB of off-chip low-cost low-power HyperRAM directly to the CPU. At the same time, it integrates a fully programmable energy- and area-efficient multi-core cluster of RV32 cores optimized for general-purpose DSP as well as reduced- and mixed-precision ML. To the best of the authors' knowledge, it is the first silicon prototype of a ULP SoC coupling the RV64 and RV32 cores in a heterogeneous host+accelerator architecture fully based on the RISC-V ISA. We demonstrate the capabilities of the proposed SoC on a wide range of benchmarks relevant to nano-UAV applications. The cluster can deliver up to 90GOp/s and up to 1.8TOp/s/W on 2-bit integer kernels and up to 7.9GFLOp/s and up to 150GFLOp/s/W on 16-bit FP kernels.

Parallelizing Optical Flow Estimation on an Ultra-Low Power RISC-V Cluster for Nano-UAV Navigation

Low Latency Visual Inertial Odometry with On-Sensor Accelerated Optical Flow for Resource-Constrained UAVs

Dynamic Illumination Optical Flow Computing for Sensing Multiple Mobile Robots from a Drone.

A Fast and Accurate Optical Flow Camera for Resource-Constrained Edge Applications

NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices

Parallel Implementation of Dense Optical Flow Computation on Many-Core Processor.

Improving Autonomous Nano-Drones Performance via Automated End-to-End Optimization and Deployment of DNNs

A Heterogeneous RISC-V based SoC for Secure Nano-UAV Navigation

An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs

Ultrafast vision perception by neuromorphic optical flow

EdgeFlowNet: 100FPS@1W Dense Optical Flow For Tiny Mobile Robots

High-Speed Scene Flow on Embedded Commercial Off-the-Shelf Systems.

NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices

A Low-Cost and Robust Optical Flow Cmos Camera for Velocity Estimation

FPGA Accelerated Real-time Recurrent All-Pairs Field Transforms for Optical Flow

Efficient Optical flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone

Time-of-Travel Methods for Measuring Optical Flow on Board a Micro Flying Robot

Fast Optical Flow Estimation Without Parallel Architectures.

Fusing Stereopsis & Corner Sparse Optical Flow for Real-time Obstacle Avoidance of Unmanned Aerial Vehicle

A Real-Time and Efficient Optical Flow Tracking Accelerator on FPGA Platform

Fully Onboard AI-powered Human-Drone Pose Estimation on Ultra-low Power Autonomous Flying Nano-UAVs