Abstract:To efficiently deploy state-of-the-art deep neural network (DNN) workloads with growing computational intensity and structural complexity, scalable DNN accelerators have been proposed in recent years, which are featured by multitensor engines and distributed on-chip buffers. Such spatial architectures have significantly expanded scheduling space in terms of parallelism and data reuse potentials, which demands for delicate workload orchestration. Previous works on DNN's hardware mapping problem mainly focus on operator-level loop transformation for single array, which are insufficient for this new challenge. Resource partitioning methods for multi-engines such as CNN-partition and inter-layer pipelining have been studied. However, their intrinsic disadvantages of workload unbalance and pipeline delay still prevent scalable accelerators from releasing full potentials. In this paper, we propose atomic dataflow, a novel graph-level scheduling and mapping approach developed for DNN inference. Instead of partitioning hardware resources into fixed regions and binding each DNN layer to a certain region sequentially, atomic dataflow schedules the DNN computation graph in workload-specific granularity (atoms) to ensure PE-array utilization, supports flexible atom ordering to exploit parallelism, and orchestrates atom-engine mapping to optimize data reuse between spatially connected tensor engines. Firstly, we propose a simulated annealing based atomic tensor generation algorithm to minimize load unbalance. Secondly, we develop a dynamic programming based atomic DAG scheduling algorithm to systematically explore massive ordering potentials. Finally, to facilitate data locality and reduce expensive off-chip memory access, we present mapping and buffering strategies to efficiently utilize distributed on-chip storage. With an automated optimization framework being established, experimental results show significant improvements over baseline approaches in terms of performance, hardware utilization, and energy consumption.

CCASM: A Computation- and Communication-Aware Scheduling and Mapping Algorithm for NoC-Based DNN Accelerators

An Efficient Algorithm for Mapping Deep Learning Applications on the NoC Architecture

A 3D Hybrid Optical-Electrical NoC Using Novel Mapping Strategy Based DCNN Dataflow Acceleration

Memory and Computation Coordinated Mapping of DNNs Onto Complex Heterogeneous SoC.

Automatic Generation and Optimization Framework of NoC-Based Neural Network Accelerator Through Reinforcement Learning

Traffic-Aware Application Mapping for Network-on-Chip Based Multiprocessor System-on-Chip

AINNS: All-Inclusive Neural Network Scheduling via Accelerator Formalization

INDM: Chiplet-Based Interconnect Network and Dataflow Mapping for DNN Accelerators

AOME: Autonomous Optimal Mapping Exploration Using Reinforcement Learning for NoC-based Accelerators Running Neural Networks

CD-MSA: Cooperative and Deadline-Aware Scheduling for Efficient Multi-Tenancy on DNN Accelerators

A NoC-based simulator for design and evaluation of deep neural networks

Optimized Mapping Spiking Neural Networks onto Network-on-Chip.

An NoC-based CNN Accelerator for Edge Computing

A Machine Learning Mapping Algorithm for NoC Optimization

An optimized mapping algorithm based on simulated annealing for regular NoC architecture

Improved Ant Colony Algorithm Based on Task Scale in Network on Chip (noc) Mapping

Dataflow Aware Mapping of Convolutional Neural Networks Onto Many-Core Platforms With Network-on-Chip Interconnect

High-performance application mapping in network-on-chip-based multicore systems

Atomic Dataflow based Graph-Level Workload Orchestration for Scalable DNN Accelerators

NN-Baton: DNN Workload Orchestration and Chiplet Granularity Exploration for Multichip Accelerators

In-Network Accumulation: Extending the Role of NoC for DNN Acceleration