Abstract:The effectiveness of deep neural networks (DNN) in vision, speech, and language processing has prompted a tremendous demand for energy-efficient high-performance DNN inference systems. Due to the increasing memory intensity of most DNN workloads, main memory can dominate the system's energy consumption and stall time. One effective way to reduce the energy consumption and increase the performance of DNN inference systems is by using approximate memory, which operates with reduced supply voltage and reduced access latency parameters that violate standard specifications. Using approximate memory reduces reliability, leading to higher bit error rates. Fortunately, neural networks have an intrinsic capacity to tolerate increased bit errors. This can enable energy-efficient and high-performance neural network inference using approximate DRAM devices. Based on this observation, we propose EDEN, a general framework that reduces DNN energy consumption and DNN evaluation latency by using approximate DRAM devices, while strictly meeting a user-specified target DNN accuracy. EDEN relies on two key ideas: 1) retraining the DNN for a target approximate DRAM device to increase the DNN's error tolerance, and 2) efficient mapping of the error tolerance of each individual DNN data type to a corresponding approximate DRAM partition in a way that meets the user-specified DNN accuracy requirements. We evaluate EDEN on multi-core CPUs, GPUs, and DNN accelerators with error models obtained from real approximate DRAM devices. For a target accuracy within 1% of the original DNN, our results show that EDEN enables 1) an average DRAM energy reduction of 21%, 37%, 31%, and 32% in CPU, GPU, and two DNN accelerator architectures, respectively, across a variety of DNNs, and 2) an average (maximum) speedup of 8% (17%) and 2.7% (5.5%) in CPU and GPU architectures, respectively, when evaluating latency-bound DNNs.

Estimating energy consumption of neural networks with joint Structure–Device encoding

Survey on Energy-Efficient Deep Neural Networks for Computer Vision

Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices

On-Device Deep Learning: Survey on Techniques Improving Energy Efficiency of DNNs

Mixed Precision Neural Architecture Search for Energy Efficient Deep Learning.

Towards Energy-Preserving Natural Language Understanding with Spiking Neural Networks

An Ensemble Energy Consumption Prediction Model for Industrial Serial-Robot

Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations

Energy Consumption of Neural Networks on NVIDIA Edge Boards: an Empirical Model

A methodological framework for optimizing the energy consumption of deep neural networks: a case study of a cyber threat detector

NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks

A Unified Evaluation Framework for Spiking Neural Network Hardware Accelerators Based on Emerging Non-Volatile Memory Devices

FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices

Arch2End: Two-Stage Unified System-Level Modeling for Heterogeneous Intelligent Devices

BPNS: a Fast and Accurate Power Estimation Model for Embedded Devices

EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAM

Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing

Energy-Constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Masking

Neural network scoring for efficient computing

Improving QoE of Deep Neural Network Inference on Edge Devices: A Bandit Approach