Abstract:Convolutional Neural Networks (CNNs) have achieved excellent performance on various artificial intelligence (AI) applications, while a higher demand on energy efficiency is required for future AI. Resistive Random-Access Memory (RRAM)-based computing system provides a promising solution to energy-efficient neural network training. However, it's difficult to support high-precision CNN in RRAM-based hardware systems. Firstly, multi-bit digital-analog interfaces will take up most energy overhead of the whole system. Secondly, it's difficult to write the RRAM to expected resistance states accurately; only low-precision numbers can be represented. To enable CNN training based on RRAM, we propose a low-bitwidth CNN training method, using low-bitwidth convolution outputs (CO), activations (A), weights (W) and gradients (G) to train CNN models based on RRAM. Furthermore, we design a system to implement the training algorithms. We explore the accuracy under different bitwidth combinations of (A,CO,W,G), and propose a practical tradeoff between accuracy and energy overhead. Our experiments demonstrate that the proposed system perform well on low-bitwidth CNN training tasks. For example, training LeNet-5 with 4-bit convolution outputs, 4-bit weights, 4-bit activations and 4-bit gradients on MNIST can still achieve 97.67% accuracy. Moreover, the proposed system can achieve 23.0X higher energy efficiency than GPU when processing the training task of LeNet-5, and 4.4X higher energy efficiency when processing the training task of ResNet-20.

24.4 Sandwich-RAM: an Energy-Efficient In-Memory BWN Architecture with Pulse-Width Modulation

A Low-Power In-Memory Multiplication and Accumulation Array with Modified Radix-4 Input and Canonical Signed Digit Weights

A 3.89-Gops/mw Scalable Recurrent Neural Network Processor with Improved Efficiency on Memory and Computation

Accelerating Low Bit-Width Deep Convolution Neural Network in MRAM.

An area and energy efficient design of domain-wall memory-based deep convolutional neural networks using stochastic computing

Binary Convolutional Neural Network on RRAM.

Training Low Bitwidth Convolutional Neural Network on RRAM

24.5 A Twin-8T SRAM Computation-In-Memory Macro for Multiple-Bit CNN-Based Machine Learning

BASER: Bit-wise Approximate Compressor Configurable In-SRAM-computing for Energy-Efficient Neural Network Acceleration with Data-aware Weight Remapping Method

RRAM-DNN: an RRAM and Model-Compression Empowered All-Weights-On-Chip DNN Accelerator

IMCE: Energy-efficient Bit-Wise In-Memory Convolution Engine for Deep Neural Network

An ADC-less RRAM-based Computing-in-Memory Macro with Binary CNN for Efficient Edge AI

Area Efficient Pattern Representation of Binary Neural Networks on RRAM

A Configurable Computing-in-Memory Structure Based on Convolutional Neural Network

CAP-RAM: A Charge-Domain In-Memory Computing 6T-SRAM for Accurate and Precision-Programmable CNN Inference

BR-CIM: an Efficient Binary Representation Computation-In-Memory Design

Switched by input: power efficient structure for RRAM-based convolutional neural network.

Memory System Designed for Multiply-Accumulate (MAC) Engine Based on Stochastic Computing.

An Energy-Efficient Mixed-Bit CNN Accelerator With Column Parallel Readout for ReRAM-Based In-Memory Computing

Parapim: A Parallel Processing-In-Memory Accelerator For Binary-Weight Deep Neural Networks

Nonvolatile Binary CNN Accelerator with Extremely Low Standby Power Using RRAM for IoT Applications