Abstract:Recent advancements in quantization and mixed-precision approaches offers substantial opportunities to improve the speed and energy efficiency of Neural Networks (NN). Research has shown that individual parameters with varying low precision, can attain accuracies comparable to full-precision counterparts. However, modern embedded microprocessors provide very limited support for mixed-precision NNs regarding both Instruction Set Architecture (ISA) extensions and their hardware design for efficient execution of mixed-precision operations, i.e., introducing several performance bottlenecks due to numerous instructions for data packing and unpacking, arithmetic unit under-utilizations etc. In this work, we bring together, for the first time, ISA extensions tailored to mixed-precision hardware optimizations, targeting energy-efficient DNN inference on leading RISC-V CPU architectures. To this end, we introduce a hardware-software co-design framework that enables cooperative hardware design, mixed-precision quantization, ISA extensions and inference in cycle-accurate emulations. At hardware level, we firstly expand the ALU unit within our proof-of-concept micro-architecture to support configurable fine grained mixed-precision arithmetic operations. Subsequently, we implement multi-pumping to minimize execution latency, with an additional soft SIMD optimization applied for 2-bit operations. At the ISA level, three distinct MAC instructions are encoded extending the RISC-V ISA, and exposed up to the compiler level, each corresponding to a different mixed-precision operational mode. Our extensive experimental evaluation over widely used DNNs and datasets, such as CIFAR10 and ImageNet, demonstrates that our framework can achieve, on average, 15x energy reduction for less than 1% accuracy loss and outperforms the ISA-agnostic state-of-the-art RISC-V cores.

RTPE: A High Energy Efficiency Inference Processor with RISC-V Based Transformation Mechanism

RCPE: an Excellent Performance Training Processor with RISC-V Based Compression Mechanism

SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference

TPE: A High-Performance Edge-Device Inference with Multi-level Transformational Mechanism

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference

A Heterogeneous RISC-V Processor for Efficient DNN Application in Smart Sensing System

A 3.89-Gops/mw Scalable Recurrent Neural Network Processor with Improved Efficiency on Memory and Computation

A Precision-Scalable RISC-V DNN Processor with On-Device Learning Capability at the Extreme Edge

RISC-V R-Extension: Advancing Efficiency with Rented-Pipeline for Edge DNN Processing

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

XpulpNN: Enabling Energy Efficient and Flexible Inference of Quantized Neural Network on RISC-V based IoT End Nodes

Xuantie-910: A Commercial Multi-Core 12-Stage Pipeline Out-of-Order 64-Bit High Performance RISC-V Processor with Vector Extension : Industrial Product

A 3 TOPS/W RISC-V Parallel Cluster for Inference of Fine-Grain Mixed-Precision Quantized Neural Networks

PL-NPU: an Energy-Efficient Edge-Device DNN Training Processor with Posit-Based Logarithm-Domain Computing

A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference

xTern: Energy-Efficient Ternary Neural Network Inference on RISC-V-Based Edge Systems

RISC-V Toolchain and Agile Development based Open-source Neuromorphic Processor

RDCIM: RISC-V Supported Full-Digital Computing-in-Memory Processor With High Energy Efficiency and Low Area Overhead

CPE: an Energy-Efficient Edge-Device Training with Multi-dimensional Compression Mechanism

Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operations