Abstract:Multiple deep neural networks (DNNs) are increasingly used in real-world intelligent applications, such as intelligent robotics and autonomous vehicles to collectively complete complicated tasks running on edge devices. Because each layer of the subtasks prefers a distinct dataflow due to the heterogeneity in shape and scale of the network layers, a variable dataflow approach on the DNN accelerators is urgently required. On DNN accelerators that enable multiple dataflows, however, we detect a dimension mismatch between parallel processing under the dataflow approach and linear data memory arrangement. When multiple DNN tasks share partial features or weights, the issue is further exacerbated. During processing, this mismatch causes a sluggish data supply from both off-chip and on-chip memory. Consequently, the overall throughput, performance, and energy efficiency suffer since DNN models are sensitive to data density. In this work, we reveal the mechanism behind this data dimension mismatch and present a series of metrics that quantify the influence on system performance. On this foundation, we offer a framework that tracks the data tensor dimension conversion and employs a flexible data arrangement over multi-DNN computation to adapt to dataflow variability. An accelerator architecture named data arrangement multi-DNN accelerator (DARMA) that features a data arrangement and distribution circuit and hierarchical memory for data dimension conversion is also presented. Since the mismatch is mitigated, the suggested accelerator outperforms current accelerators in terms of bandwidth and processing unit utilization. Through tests on VR/AR, MLperf, and other multitask applications, the evaluation results show that the proposed architecture provides both energy-efficiency and throughput improvements.

DUET: Boosting Deep Neural Network Efficiency on Dual-Module Architecture

DaDianNao: A Machine-Learning Supercomputer

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

Memory Bandwidth and Energy Efficiency Optimization of Deep Convolutional Neural Network Accelerators.

Memory-Computing Decoupling: A DNN Multitasking Accelerator with Adaptive Data Arrangement.

Dual-module Inference for Efficient Recurrent Neural Networks

INDM: Chiplet-Based Interconnect Network and Dataflow Mapping for DNN Accelerators

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

A Data-Driven Asynchronous Neural Network Accelerator

Dynamic Path Based DNN Synergistic Inference Acceleration in Edge Computing Environment.

Optimizing DNN Inference on Multi-Accelerator SoCs at Training-time

Energy-Efficient Accelerator Design for Deformable Convolution Networks

Software-defined Design Space Exploration for an Efficient DNN Accelerator Architecture

Layer-Wise Mixed-Modes CNN Processing Architecture With Double-Stationary Dataflow and Dimension-Reshape Strategy

A Parallel Loading Based Accelerator for Convolution Neural Network

A High Efficient Architecture for Convolution Neural Network Accelerator

A Precision-Scalable Deep Neural Network Accelerator with Activation Sparsity Exploitation

An Efficient Sparse-Aware Summation Optimization Strategy for DNN Accelerator

An Energy-Efficient Near-Data Processing Accelerator for DNNs that Optimizes Data Accesses

Accelerating Deep Neural Networks by Combining Block-Circulant Matrices and Low-Precision Weights