Abstract:Multiple deep neural networks (DNNs) are increasingly used in real-world intelligent applications, such as intelligent robotics and autonomous vehicles to collectively complete complicated tasks running on edge devices. Because each layer of the subtasks prefers a distinct dataflow due to the heterogeneity in shape and scale of the network layers, a variable dataflow approach on the DNN accelerators is urgently required. On DNN accelerators that enable multiple dataflows, however, we detect a dimension mismatch between parallel processing under the dataflow approach and linear data memory arrangement. When multiple DNN tasks share partial features or weights, the issue is further exacerbated. During processing, this mismatch causes a sluggish data supply from both off-chip and on-chip memory. Consequently, the overall throughput, performance, and energy efficiency suffer since DNN models are sensitive to data density. In this work, we reveal the mechanism behind this data dimension mismatch and present a series of metrics that quantify the influence on system performance. On this foundation, we offer a framework that tracks the data tensor dimension conversion and employs a flexible data arrangement over multi-DNN computation to adapt to dataflow variability. An accelerator architecture named data arrangement multi-DNN accelerator (DARMA) that features a data arrangement and distribution circuit and hierarchical memory for data dimension conversion is also presented. Since the mismatch is mitigated, the suggested accelerator outperforms current accelerators in terms of bandwidth and processing unit utilization. Through tests on VR/AR, MLperf, and other multitask applications, the evaluation results show that the proposed architecture provides both energy-efficiency and throughput improvements.

Memory and Computation Coordinated Mapping of DNNs Onto Complex Heterogeneous SoC.

DaDianNao: A Machine-Learning Supercomputer

Memory-Computing Decoupling: A DNN Multitasking Accelerator with Adaptive Data Arrangement.

CCASM: A Computation- and Communication-Aware Scheduling and Mapping Algorithm for NoC-Based DNN Accelerators

A 3D Hybrid Optical-Electrical NoC Using Novel Mapping Strategy Based DCNN Dataflow Acceleration

NN-Baton: DNN Workload Orchestration and Chiplet Granularity Exploration for Multichip Accelerators

Cocco: Hardware-Mapping Co-Exploration towards Memory Capacity-Communication Optimization

Memory optimization of CNN Heterogeneous Multi-Core Architecture

A Customized NoC Architecture to Enable Highly Localized Computing-On-the-Move DNN Dataflow

M2M: A Fine-Grained Mapping Framework to Accelerate Multiple DNNs on a Multi-Chiplet Architecture

Automatic Mapping of Heterogeneous DNN Models on Adaptive Multi-Accelerator Systems

Memory Access Aware Mapping for Networks-on-Chip

INDM: Chiplet-Based Interconnect Network and Dataflow Mapping for DNN Accelerators

Overcoming Data Transfer Bottlenecks in FPGA-based DNN Accelerators Via Layer Conscious Memory Management

Dataflow Aware Mapping of Convolutional Neural Networks Onto Many-Core Platforms With Network-on-Chip Interconnect

CMDS: Cross-layer Dataflow Optimization for DNN Accelerators Exploiting Multi-bank Memories

Memory Bandwidth and Energy Efficiency Optimization of Deep Convolutional Neural Network Accelerators.

Multi-Objective Hardware-Mapping Co-Optimisation for Multi-DNN Workloads on Chiplet-based Accelerators

Optimizing DNN Inference on Multi-Accelerator SoCs at Training-time

Traffic-Aware Application Mapping for Network-on-Chip Based Multiprocessor System-on-Chip

HSC: A Hybrid Spin/CMOS Logic Based In-Memory Engine with Area-Efficient Mapping Strategy.