Abstract:Convolutional Neural Networks (CNNs) have made breakthroughs in various fields, while the energy consumption becomes enormous. Processing-In-Memory (PIM) architectures based on emerging non-volatile memory (e.g., Resistive Random Access Memory, RRAM) have demonstrated great potential in improving the energy efficiency of CNN computing. However, there is still much room for improvement in the energy efficiency of existing PIM architectures. On the one hand, current work shows that high resolution Analog-to-Digital Converters (ADCs) are required for maintaining computing accuracy, but they dominate more than 60% energy consumption of the entire system, damaging the energy efficiency benefits of PIM. On the other hand, the characteristic of computing in the analog domain in PIM accelerators leads to the computing energy consumption is influenced by the specific input and weight values. However, as far as we know, there is no energy efficiency optimization method based on this characteristic in existing work. To solve these problems, in this paper, we propose an energy-efficient quantized and regularized training framework for PIM accelerators, which consists of a PIM-based non-uniform activation quantization scheme and an energy-aware weight regularization method. The proposed framework can improve the energy efficiency of PIM architectures by reducing the ADC resolution requirements and training low energy consumption CNN models for PIM, with little accuracy loss. The experimental results show that the proposed training framework can reduce the resolution of ADCs by 2 bits and the computing energy consumption in the analog domain by 35%. The energy efficiency, therefore, can be enhanced by $3.4 \times$ in our proposed training framework.

High Area/Energy Efficiency RRAM CNN Accelerator with Pattern-Pruning-Based Weight Mapping Scheme

RRAM Based Buffer Design for Energy Efficient CNN Accelerator.

Mixed Size Crossbar Based RRAM CNN Accelerator with Overlapped Mapping Method

AEPE: an Area and Power Efficient RRAM Crossbar-Based Accelerator for Deep CNNs

Area Efficient Pattern Representation of Binary Neural Networks on RRAM

A Configurable Multi-Precision CNN Computing Framework Based on Single Bit RRAM

Training Low Bitwidth Convolutional Neural Network on RRAM

Switched by input: power efficient structure for RRAM-based convolutional neural network.

Low Bit-Width Convolutional Neural Network on RRAM

SNrram: an Efficient Sparse Neural Network Computation Architecture Based on Resistive Random-Access Memory.

An Energy-Efficient Mixed-Bit CNN Accelerator With Column Parallel Readout for ReRAM-Based In-Memory Computing

Efficient and Robust RRAM-Based Convolutional Weight Mapping With Shifted and Duplicated Kernel

Learning the sparsity for ReRAM - mapping and pruning sparse neural network for ReRAM based accelerator.

NAND-SPIN-based processing-in-MRAM architecture for convolutional neural network acceleration

RRAM-DNN: an RRAM and Model-Compression Empowered All-Weights-On-Chip DNN Accelerator

A RRAM Based Max-Pooling Scheme for Convolutional Neural Network

Reducing SRAM Reading Power with Column Data Segment and Weights Correlation Enhancement for CNN Processing.

RRAM Based Convolutional Neural Networks for High Accuracy Pattern Recognition and Online Learning Tasks

An Energy-Efficient Quantized and Regularized Training Framework for Processing-In-Memory Accelerators

An Efficient CNN Accelerator for Pattern-Compressed Sparse Neural Networks on FPGA

An Energy-Efficient Gain-Cell Embedded DRAM Design with Weight Encoding for CNN Applications.