Abstract:The novel Computing-In-Memory (CIM) technology has demonstrated significant potential in enhancing the performance and efficiency of convolutional neural networks (CNNs). However, due to the low precision of memory devices and data interfaces, an additional quantization step is necessary. Conventional NN quantization methods fail to account for the hardware characteristics of CIM, resulting in inferior system performance and efficiency. This paper proposes CIMQ, a hardware-efficient quantization framework designed to improve the efficiency of CIM based NN accelerators. The holistic framework focuses on the fundamental computing elements in CIM hardware: inputs, weights and outputs (or activations, weights and partial sums in NNs) with four innovative techniques. Firstly, bit-level sparsity induced activation quantization is introduced to decrease dynamic computation energy. Secondly, inspired by the unique computation paradigm of CIM, an innovative array-wise quantization granularity is proposed for weight quantization. Thirdly, partial sums are quantized with a reparametrized clipping function to reduce the required resolution of analog-to-digital converters (ADCs). Finally, to improve the accuracy of quantized neural networks (QNNs), the post-training quantization (PTQ) is enhanced with a random quantization dropping strategy. The effectiveness of the proposed framework has been demonstrated through experimental results on various NNs and datasets (CIFAR10, CIFAR100, ImageNet). In typical cases, the hardware efficiency can be improved up to 222% with a 58.97% improvement in accuracy compared to conventional quantization methods.

A Reliability-Concerned Compute-in-Memory Behavior Model for Convolutional Neural Network

CIMQ: A Hardware-Efficient Quantization Framework for Computing-In-Memory Based Neural Network Accelerators

Optimizing the Learning Accuracy in Mobile Augmented Reality Systems with CNN

Redundancy-Reduced MobileNet Acceleration on Reconfigurable Logic for ImageNet Classification

Improved JPEG Lossless Compression for Compression of Intermediate Layers in Neural Networks Based on Compute-In-Memory

IC Networks: Remodeling the Basic Unit for Convolutional Neural Networks

Reliability-Aware Training and Performance Modeling for Processing-In-Memory Systems

Intra-array Non-Idealities Modeling and Algorithm Optimization for RRAM-based Computing-in-Memory Applications

Pipeline Design of Nonvolatile-based Computing in Memory for Convolutional Neural Networks Inference Accelerators

A 2.75-to-75.9tops/w Computing-in-Memory NN Processor Supporting Set-Associate Block-Wise Zero Skipping and Ping-Pong CIM with Simultaneous Computation and Weight Updating.

A Configurable Computing-in-Memory Structure Based on Convolutional Neural Network

Accuracy and Resiliency of Analog Compute-in-Memory Inference Engines

An Adaptive Read Control Voltage Scheme for Reliability Enhancement of Flash-Based In-Memory Computing Architecture for Neural Network

Design Framework for SRAM-Based Computing-In-Memory Edge CNN Accelerators

Towards efficient allocation of graph convolutional networks on hybrid computation-in-memory architecture

Weight and Multiply-Accumulation Sparsity-Aware Non-Volatile Computing-in-Memory System

DNN+NeuroSim V2.0: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators for On-chip Training

A Sliding-Kernel Computation-In-Memory Architecture for Convolutional Neural Network

NAND-SPIN-based processing-in-MRAM architecture for convolutional neural network acceleration

Algorithm-Based Fault Tolerance for Convolutional Neural Networks

Evaluating Data Resilience in CNNs from an Approximate Memory Perspective.