Abstract:Recently, deep convolutional neural networks (CNNs) have achieved eye-catching results in various applications. However, intensive memory access of activations introduces considerable energy consumption, resulting in a great challenge for deploying CNNs on resource-constrained edge devices. Existing research utilizes dimension reduction and mixed-precision quantization separately to reduce computational complexity without paying attention to their interaction. Such naïve concatenation of different compression strategies ends up with sub-optimal performance. To develop a comprehensive compression framework, we propose an optimization system by jointly considering dimension reduction and mixed-precision quantization, which is enabled by independent group-wise learnable mixed-precision schemes. Group partitioning is guided by a well-designed automatic group partition mechanism that can distinguish compression priorities among channels, and it can deal with the trade-off between model accuracy and compressibility. Moreover, to preserve model accuracy under low bit-width quantization, we propose a dynamic bit-width searching technique to enable continuous bit-width reduction. Our experimental results show that the proposed system reaches 69.03%/70.73% with average 2.16/2.61 bits per value on Resnet18/MobileNetV2, while introducing only approximately 1% accuracy loss of the uncompressed full-precision models. Compared with individual activation compression schemes, the proposed joint optimization system reduces 55%/9% (-2.62/-0.27 bits) memory access of dimension reduction and 55%/63% (-2.60/-4.52 bits) memory access of mixed-precision quantization, respectively, on Resnet18/ MobileNetV2 with comparable or even higher accuracy.

Approximate On-Chip Memory Optimization Method For Deep Residual Networks

Evaluating Data Resilience in CNNs from an Approximate Memory Perspective.

Overcoming Memory Constraint for Improved Target Classification Performance on Embedded Deep Learning Systems

Double-Shift: A Low-Power DNN Weights Storage and Access Framework Based on Approximate Decomposition and Quantization

Optimization of GPU Memory Usage for Training Deep Neural Networks.

Memory-efficient Deep Learning Inference with Incremental Weight Loading and Data Layout Reorganization on Edge Systems.

AppSRAM: Incremental Retraining Method to Compensate for SRAM Failure in Voltage-Scaling Embedded CNN System

Optimizing Off-Chip Memory Access for Deep Neural Network Accelerator

A Multiply-Less Approximate SRAM Compute-In-Memory Macro for Neural-Network Inference

Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation

Optimizing for In-memory Deep Learning with Emerging Memory Technology

A Hybrid Precision Low Power Computing-in-memory Architecture for Neural Networks

Energy-efficient Neural Networks Using Approximate Computation Reuse.

A Multi-accuracy-Level Approximate Memory Architecture Based on Data Significance Analysis

Array-level Boosting Method with Spatial Extended Allocation to Improve the Accuracy of Memristor Based Computing-in-memory Chips

ReDistill: Residual Encoded Distillation for Peak Memory Reduction

A Hybrid RRAM-SRAM Computing-In-Memory Architecture for Deep Neural Network Inference-Training Edge Acceleration

DNN Memory Footprint Reduction via Post-Training Intra-Layer Multi-Precision Quantization

Improving Memory Utilization in Convolutional Neural Network Accelerators

INA: Incremental Network Approximation Algorithm for Limited Precision Deep Neural Networks

Joint Optimization of Dimension Reduction and Mixed-Precision Quantization for Activation Compression of Neural Networks