Abstract:To efficiently deploy state-of-the-art deep neural network (DNN) workloads with growing computational intensity and structural complexity, scalable DNN accelerators have been proposed in recent years, which are featured by multitensor engines and distributed on-chip buffers. Such spatial architectures have significantly expanded scheduling space in terms of parallelism and data reuse potentials, which demands for delicate workload orchestration. Previous works on DNN's hardware mapping problem mainly focus on operator-level loop transformation for single array, which are insufficient for this new challenge. Resource partitioning methods for multi-engines such as CNN-partition and inter-layer pipelining have been studied. However, their intrinsic disadvantages of workload unbalance and pipeline delay still prevent scalable accelerators from releasing full potentials. In this paper, we propose atomic dataflow, a novel graph-level scheduling and mapping approach developed for DNN inference. Instead of partitioning hardware resources into fixed regions and binding each DNN layer to a certain region sequentially, atomic dataflow schedules the DNN computation graph in workload-specific granularity (atoms) to ensure PE-array utilization, supports flexible atom ordering to exploit parallelism, and orchestrates atom-engine mapping to optimize data reuse between spatially connected tensor engines. Firstly, we propose a simulated annealing based atomic tensor generation algorithm to minimize load unbalance. Secondly, we develop a dynamic programming based atomic DAG scheduling algorithm to systematically explore massive ordering potentials. Finally, to facilitate data locality and reduce expensive off-chip memory access, we present mapping and buffering strategies to efficiently utilize distributed on-chip storage. With an automated optimization framework being established, experimental results show significant improvements over baseline approaches in terms of performance, hardware utilization, and energy consumption.

COCOA: Content-Oriented Configurable Architecture Based on Highly-Adaptive Data Transmission Networks.

Exploiting Concurrency for Opportunistic Forwarding in Duty-Cycled IoT Networks

DaDianNao: A Machine-Learning Supercomputer

A Physical-Aware Framework for Memory Network Design Space Exploration

Cocco: Hardware-Mapping Co-Exploration towards Memory Capacity-Communication Optimization

A More Efficient COPE Architecture for Network Coding in Multihop Wireless Networks.

CooCo: A Collaborative Offloading and Resource Configuration Algorithm in Edge Networks

A Customized NoC Architecture to Enable Highly Localized Computing-On-the-Move DNN Dataflow

An Optimization Toolchain Design Of Deep Learning Deployment Based On Heterogeneous Computing Platform

Novel many-core architecture design for real-time image processing

Flow-oriented Network Coding Architecture for Multihop Wireless Networks.

Cross-object Coding and Allocation (COCA) for Distributed Storage Systems

Memory and Computation Coordinated Mapping of DNNs Onto Complex Heterogeneous SoC.

Joint Optimization of Data Transfer and Co-Execution for DNN in Edge Computing

COAC: Cross-layer Optimization of Accelerator Configurability for Efficient CNN Processing

Adaptive Privacy-Preserving Coded Computing With Hierarchical Task Partitioning

CoCo: Exploiting Correlated Core for Energy Efficient Dissemination in Wireless Sensor Networks.

Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

Atomic Dataflow based Graph-Level Workload Orchestration for Scalable DNN Accelerators

Domino: A Tailored Network-on-Chip Architecture to Enable Highly Localized Inter- and Intra-Memory DNN Computing