Abstract:The effectiveness of deep neural networks (DNN) in vision, speech, and language processing has prompted a tremendous demand for energy-efficient high-performance DNN inference systems. Due to the increasing memory intensity of most DNN workloads, main memory can dominate the system's energy consumption and stall time. One effective way to reduce the energy consumption and increase the performance of DNN inference systems is by using approximate memory, which operates with reduced supply voltage and reduced access latency parameters that violate standard specifications. Using approximate memory reduces reliability, leading to higher bit error rates. Fortunately, neural networks have an intrinsic capacity to tolerate increased bit errors. This can enable energy-efficient and high-performance neural network inference using approximate DRAM devices. Based on this observation, we propose EDEN, a general framework that reduces DNN energy consumption and DNN evaluation latency by using approximate DRAM devices, while strictly meeting a user-specified target DNN accuracy. EDEN relies on two key ideas: 1) retraining the DNN for a target approximate DRAM device to increase the DNN's error tolerance, and 2) efficient mapping of the error tolerance of each individual DNN data type to a corresponding approximate DRAM partition in a way that meets the user-specified DNN accuracy requirements. We evaluate EDEN on multi-core CPUs, GPUs, and DNN accelerators with error models obtained from real approximate DRAM devices. For a target accuracy within 1% of the original DNN, our results show that EDEN enables 1) an average DRAM energy reduction of 21%, 37%, 31%, and 32% in CPU, GPU, and two DNN accelerator architectures, respectively, across a variety of DNNs, and 2) an average (maximum) speedup of 8% (17%) and 2.7% (5.5%) in CPU and GPU architectures, respectively, when evaluating latency-bound DNNs.

Evaluating Data Resilience in CNNs from an Approximate Memory Perspective.

Approximate On-Chip Memory Optimization Method For Deep Residual Networks

AxR-NN: Approximate Computation Reuse for Energy-Efficient Convolutional Neural Networks

A Multi-accuracy-Level Approximate Memory Architecture Based on Data Significance Analysis

ReD-CaNe: A Systematic Methodology for Resilience Analysis and Design of Capsule Networks under Approximations

AppSRAM: Incremental Retraining Method to Compensate for SRAM Failure in Voltage-Scaling Embedded CNN System

An Energy-Efficient Quantized and Regularized Training Framework for Processing-In-Memory Accelerators

A Configurable Multi-Precision CNN Computing Framework Based on Single Bit RRAM

In-Place Zero-Space Memory Protection for CNN

Concrete: A Per-layer Configurable Framework for Evaluating DNN with Approximate Operators

EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAM

Approximate Processing Element Design and Analysis for the Implementation of CNN Accelerators

A Reliability-Concerned Compute-in-Memory Behavior Model for Convolutional Neural Network

Accuracy and Resiliency of Analog Compute-in-Memory Inference Engines

Algorithm-Based Fault Tolerance for Convolutional Neural Networks

Reliable and Energy Efficient MLC STT-RAM Buffer for CNN Accelerators

CIMQ: A Hardware-Efficient Quantization Framework for Computing-In-Memory Based Neural Network Accelerators

Bit-Aware Fault-Tolerant Hybrid Retraining and Remapping Schemes for RRAM-Based Computing-in-Memory Systems

ReDy: A Novel ReRAM-centric Dynamic Quantization Approach for Energy-efficient CNN Inference

LR-CNN: Lightweight Row-centric Convolutional Neural Network Training for Memory Reduction

Exposing Reliability Degradation and Mitigation in Approximate DNNs under Permanent Faults