Abstract:Multiple deep neural networks (DNNs) are increasingly used in real-world intelligent applications, such as intelligent robotics and autonomous vehicles to collectively complete complicated tasks running on edge devices. Because each layer of the subtasks prefers a distinct dataflow due to the heterogeneity in shape and scale of the network layers, a variable dataflow approach on the DNN accelerators is urgently required. On DNN accelerators that enable multiple dataflows, however, we detect a dimension mismatch between parallel processing under the dataflow approach and linear data memory arrangement. When multiple DNN tasks share partial features or weights, the issue is further exacerbated. During processing, this mismatch causes a sluggish data supply from both off-chip and on-chip memory. Consequently, the overall throughput, performance, and energy efficiency suffer since DNN models are sensitive to data density. In this work, we reveal the mechanism behind this data dimension mismatch and present a series of metrics that quantify the influence on system performance. On this foundation, we offer a framework that tracks the data tensor dimension conversion and employs a flexible data arrangement over multi-DNN computation to adapt to dataflow variability. An accelerator architecture named data arrangement multi-DNN accelerator (DARMA) that features a data arrangement and distribution circuit and hierarchical memory for data dimension conversion is also presented. Since the mismatch is mitigated, the suggested accelerator outperforms current accelerators in terms of bandwidth and processing unit utilization. Through tests on VR/AR, MLperf, and other multitask applications, the evaluation results show that the proposed architecture provides both energy-efficiency and throughput improvements.

Cocco: Hardware-Mapping Co-Exploration towards Memory Capacity-Communication Optimization

DaDianNao: A Machine-Learning Supercomputer

A Physical-Aware Framework for Memory Network Design Space Exploration

Memory and Computation Coordinated Mapping of DNNs Onto Complex Heterogeneous SoC.

Exploration of Memory Access Optimization for FPGA-based 3D CNN Accelerator.

COCOA: Content-Oriented Configurable Architecture Based on Highly-Adaptive Data Transmission Networks.

Memory optimization of CNN Heterogeneous Multi-Core Architecture

Study on Optimization Techniques for Memory Accesses of CUDA Parallel Programs

NN-Baton: DNN Workload Orchestration and Chiplet Granularity Exploration for Multichip Accelerators

CMDS: Cross-layer Dataflow Optimization for DNN Accelerators Exploiting Multi-bank Memories

Memory-Computing Decoupling: A DNN Multitasking Accelerator with Adaptive Data Arrangement.

Memory Access Aware Mapping for Networks-on-Chip

Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators

ZigZag: A Memory-Centric Rapid DNN Accelerator Design Space Exploration Framework

MAGIS: Memory Optimization Via Coordinated Graph Transformation and Scheduling for DNN

Overcoming Data Transfer Bottlenecks in FPGA-based DNN Accelerators Via Layer Conscious Memory Management

Memory Bandwidth and Energy Efficiency Optimization of Deep Convolutional Neural Network Accelerators.

A Design Space Exploration Method For On-Chip Memory System Based On Task Scheduling

System and Design Technology Co-optimization of SOT-MRAM for High-Performance AI Accelerator Memory System

AOME: Autonomous Optimal Mapping Exploration Using Reinforcement Learning for NoC-based Accelerators Running Neural Networks

Gemini: Mapping and Architecture Co-exploration for Large-scale DNN Chiplet Accelerators