Abstract:In the field of edge computing, quantizing convolutional neural networks (CNNs) using extremely low bit widths can significantly alleviate the associated storage and computational burdens in embedded hardware, thereby improving computational efficiency. However, such quantization also presents a challenge related to substantial decreases in detection accuracy. This paper proposes an innovative method, called Adaptive Global Power-of-Two Ternary Quantization Based on Unfixed Boundary Thresholds (APTQ). APTQ achieves adaptive quantization by quantizing each filter into two binary subfilters represented as power-of-two values, thereby addressing the accuracy degradation caused by a lack of expression ability of low-bit-width weight values and the contradiction between fixed quantization boundaries and the uneven actual weight distribution. It effectively reduces the accuracy loss while at the same time presenting strong hardware-friendly characteristics because of the power-of-two quantization. This paper extends the APTQ algorithm to propose the APQ quantization algorithm, which can adapt to arbitrary quantization bit widths. Furthermore, this paper designs dedicated edge deployment convolutional computation modules for the obtained quantized models. Through quantization comparison experiments with multiple commonly used CNN models utilized on the CIFAR10, CIFAR100, and Mini-ImageNet data sets, it is verified that the APTQ and APQ algorithms possess better accuracy performance than most state-of-the-art quantization algorithms and can achieve results with very low accuracy loss in certain CNNs (e.g., the accuracy loss of the APTQ ternary ResNet-56 model on CIFAR10 is 0.13%). The dedicated convolutional computation modules enable the corresponding quantized models to occupy fewer on-chip hardware resources in edge chips, thereby effectively improving computational efficiency. This adaptive CNN quantization method, combined with the power-of-two quantization results, strikes a balance between the quantization accuracy performance and deployment efficiency in embedded hardware. As such, valuable insights for the industrial edge computing domain can be gained.

Cloud-Edge Inference under Communication Constraints: Data Quantization and Early Exit.

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

A privacy protection approach in edge-computing based on maximized dnn partition strategy with energy saving

A Survey on Deep Neural Network Partition over Cloud, Edge and End Devices

Dynamic DNN Decomposition for Lossless Synergistic Inference

An Adaptive DNN Inference Acceleration Framework with End–edge–cloud Collaborative Computing

DDPQN: An Efficient DNN Offloading Strategy in Local-Edge-Cloud Collaborative Environments

Adaptive Global Power-of-Two Ternary Quantization Algorithm Based on Unfixed Boundary Thresholds

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

Multi-Compression Scale DNN Inference Acceleration based on Cloud-Edge-End Collaboration

HiTDL: High-Throughput Deep Learning Inference at the Hybrid Mobile Edge

Low Power Inference for On-Device Visual Recognition with a Quantization-Friendly Solution.

CDC: Classification Driven Compression for Bandwidth Efficient Edge-Cloud Collaborative Deep Learning

Dataflow-Based Joint Quantization for Deep Neural Networks

Inference Time Optimization Using BranchyNet Partitioning

Quantization Networks

PArtNNer: Platform-agnostic Adaptive Edge-Cloud DNN Partitioning for minimizing End-to-End Latency

DQI: A Dynamic Quantization Method for Efficient Convolutional Neural Network Inference Accelerators

Distributed Assignment With Load Balancing for DNN Inference at the Edge

A Quantization-Friendly Separable Convolution for MobileNets.