Abstract:As one of the prevailing deep neural networks compression techniques, layer-wise mixed-precision quantization (MPQ) strikes a better balance between accuracy and efficiency than uniform quantization schemes. However, existing MPQ strategies either lack hardware awareness or incur huge computation costs, limiting their deployment at the edge. Additionally, researchers usually make a one-time decision between post-training quantization (PTQ) and quantization-aware training (QAT) based on the quantized bit-width or hardware requirements. In this paper, we propose the tight integration of versatile MPQ inference units supporting INT2-INT8 and INT16 precisions, which feature a hierarchical multiplier architecture, into a RISC-V processor pipeline through micro-architecture and Instruction Set Architecture (ISA) co-design. Synthesized with a 14nm technology, the design delivers a speedup of 15.50× to 47.67× over the baseline RV64IMA core when running a single convolution layer kernel and achieves up to 2.86 GOPS performance. This work also achieves an energy efficiency at 20.51 TOPS/W, which not only exceeds contemporary state-of-the-art MPQ hardware solutions at the edge, but also marks a significant advancement in the field. We also propose a novel MPQ search algorithm that incorporates both hardware awareness and training necessity. The algorithm samples layer-wise sensitivities using a set of newly proposed metrics and runs a heuristics search. Evaluation results show that this search algorithm achieves 2.2%∼6.7% higher inference accuracy under similar hardware constraints compared to state-of-the-art MPQ strategies. Furthermore we expand the search space using a dynamic programming (DP) strategy to perform search with more fine-grained accuracy intervals and support multi-dimensional search. This further improves the inference accuracy by over 1.3% compared to a greedy-based search.

W-IQ: Wither-logic based issue queue for RISC-V superscalar out-of-order processor

Xuantie-910: A Commercial Multi-Core 12-Stage Pipeline Out-of-Order 64-Bit High Performance RISC-V Processor with Vector Extension : Industrial Product

Improving Ilp Via Fused In-Order Superscalar And Vliw Instruction Dispatch Methods

Orinoco: Ordered Issue and Unordered Commit with Non-Collapsible Queues.

On Reducing the Execution Latency of Superconducting Quantum Processors via Quantum Program Scheduling

An Energy-Efficient Instruction Scheduler Design with Two-Level Shelving and Adaptive Banking

Edge-MPQ: Layer-Wise Mixed-Precision Quantization With Tightly Integrated Versatile Inference Units for Edge Computing

ISAAQ: Ising Machine Assisted Quantum Compiler

AnnikaCore: RISC-V Architecture Processor Design and Implementation for IoT

High-Quality Iterative Logic Compiler for In-Memory SIMD Computation with Tight Coupling of Synthesis and Scheduling

A Theoretical Research on Program Instruction Level Parallelism to Guide Microprocessor Design

Run-Time Phase Prediction For A Reconfigurable Vliw Processor

COMPAQT: Compressed Waveform Memory Architecture for Scalable Qubit Control

A Low-Power Asynchronous RISC-V Processor with Propagated Timing Constraints Method

Enabling Efficient Hybrid Systolic Computation in Shared-L1-Memory Manycore Clusters

A Heterogeneous RISC-V SoC for Confidential Computing and Password Recovery

XpulpNN: Enabling Energy Efficient and Flexible Inference of Quantized Neural Network on RISC-V based IoT End Nodes

A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference

VPQC: A Domain-Specific Vector Processor for Post-Quantum Cryptography Based on RISC-V Architecture

A Reusable Architecture Design And Implementation For Inverse Quantization Of Mp3 Decoding

SIMF: Single-Instruction Multiple-Flush Mechanism for Processor Temporal Isolation