Abstract:CNN model computation on edge devices is tightly restricted to the limited resource and power budgets, which motivates the low-bit quantization technology to compress CNN models into 4-bit or lower format to reduce the model size and increase hardware efficiency. Most current low-bit quantization methods use uniform quantization that maps weight and activation values onto evenly-distributed levels, which usually results in accuracy loss due to distribution mismatch. Meanwhile, some non-uniform quantization methods propose specialized representation that can better match various distribution shapes but are usually difficult to be efficiently accelerated on hardware. In order to achieve low-bit quantization with high accuracy and hardware efficiency, this paper proposes Universal Power-of-Two (UPoT), a novel low-bit quantization method that represents values as the addition of multiple power-of-two values selected from a series of subsets. By updating the subset contents, UPoT can provide adaptive quantization levels for various distributions. For each CNN model layer, UPoT automatically searches for the optimized distribution that minimizes the quantization error. Moreover, we design an efficient accelerator system with specifically optimized power-of-two multipliers and requantization units. Evaluations show that the proposed architecture can provide high-performance CNN inference with reduced circuit area and energy, and outperforms several mainstream CNN accelerators with higher ( $8\times $ – $65\times $ ) area efficiency and ( $2\times $ – $19\times $ ) energy efficiency. Further experiments of 4/3/2-bit quantization on ResNet18/50, MobileNet_V2 and EfficientNet models show that our UPoT can achieve high model accuracy which greatly outperform other state-of-the-art low-bit quantization methods by 0.3%–6%. The results indicate that our approach provides a highly-efficient accelerator for low-bit CNN model quantization with low hardware overheads and good model accuracy.

CoopNet: Cooperative Convolutional Neural Network for Low-Power MCUs

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Low power Convolutional Neural Networks on a chip

Reconfigurable co-processor architecture with limited numerical precision to accelerate deep convolutional neural networks

MCU-MixQ: A HW/SW Co-optimized Mixed-precision Neural Network Design Framework for MCUs

ZIP-CNN: Design Space Exploration for CNN Implementation within a MCU

SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers

CMix-NN: Mixed Low-Precision CNN Library for Memory-Constrained Edge Devices

KCNN: Kernel-wise Quantization to Remarkably Decrease Multiplications in Convolutional Neural Network.

ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices

A Low-Power Hardware Architecture for Real-Time CNN Computing

Deploy Large-Scale Deep Neural Networks in Resource Constrained IoT Devices with Local Quantization Region

CNN hardware acceleration on a low-power and low-cost APSoC

An Energy-and-Area-Efficient CNN Accelerator for Universal Powers-of-Two Quantization.

ON-CNN: Low Latency and High Throughput Online Arithmetic-Based Convolutional Neural Network Accelerator

An Efficient Streaming Accelerator for Low Bit-Width Convolutional Neural Networks

Cooperative Inference with Interleaved Operator Partitioning for CNNs

Leveraging Automated Mixed-Low-Precision Quantization for tiny edge microcontrollers

Efficient Neural Network Deployment for Microcontroller

Joint Optimization of Dimension Reduction and Mixed-Precision Quantization for Activation Compression of Neural Networks

Learning Low Resource Consumption CNN through Pruning and Quantization