Abstract:Systolic array architecture has significantly accelerated deep neural networks (DNNs). A systolic array comprises multiple processing elements (PEs) that can perform multiply-accumulate (MAC). Traditionally, the systolic array can execute a certain amount of tensor data that matches the size of the systolic array simultaneously at each cycle. However, hyper-parameters of DNN models differ across each layer and result in various tensor sizes in each layer. Mapping these irregular tensors to the systolic array while fully utilizing the entire PEs in a systolic array is challenging. Furthermore, modern DNN systolic accelerators typically employ a single dataflow. However, such a dataflow isn’t optimal for every DNN model. This work proposes ReSA, a reconfigurable dataflow architecture that aims to minimize the execution time of a DNN model by mapping tiny tensors on the spatially partitioned systolic array. Unlike conventional systolic array architectures, the ReSA data path controller enables the execution of the input, weight, and output-stationary dataflow on PEs. ReSA also decomposes the coarse-grain systolic array into multiple small ones to reduce the fragmentation issue on the tensor mapping. Each small systolic sub-array unit relies on our data arbiter to dispatch tensors to each other through the simple interconnected network. Furthermore, ReSA reorders the memory access to overlap the memory load and execution stages to hide the memory latency when tackling tiny tensors. Finally, ReSA splits tensors of each layer into multiple small ones and searches for the best dataflow for each tensor on the host side. Then, ReSA encodes the predefined dataflow in our proposed instruction to notify the systolic array to switch the dataflow correctly. As a result, our optimization on the systolic array architecture achieves a geometric mean speedup of 1.87X over the weight-stationary systolic array architecture across 9 different DNN models.

Generating Systolic Array Accelerators with Reusable Blocks

On the Difficulty of Designing Processor Arrays for Deep Neural Networks

SuSy: A Programming Model for Productive Construction of High-Performance Systolic Arrays on FPGAs

Search for Optimal Systolic Arrays: A Comprehensive Automated Exploration Framework and Lessons Learned

Systolic Computing on GPUs for Productive Performance

SCALE-Sim: Systolic CNN Accelerator Simulator

Scale-out Systolic Arrays

ReDas: A Lightweight Architecture for Supporting Fine-Grained Reshaping and Multiple Dataflows on Systolic Array

ONE-SA: Enabling Nonlinear Operations in Systolic Arrays for Efficient and Flexible Neural Network Inference

Configurable Multi-directional Systolic Array Architecture for Convolutional Neural Networks

HSAS: Efficient task scheduling for large scale heterogeneous systolic array accelerator cluster

Hardware Acceleration for the Banded Smith-Waterman Algorithm with the Cycled Systolic Array

EMS: Efficient Memory Subsystem Synthesis for Spatial Accelerators

Enabling Efficient Hybrid Systolic Computation in Shared-L1-Memory Manycore Clusters

ReSA: Reconfigurable Systolic Array for Multiple Tiny DNN Tensors

Addressing the issue of processing element under-utilization in general-purpose systolic deep learning accelerators

Revealing Untapped DSP Optimization Potentials for FPGA-Based Systolic Matrix Engines

Iris: Automatic Generation of Efficient Data Layouts for High Bandwidth Utilization

A High-Performance Systolic Array Accelerator Dedicated for CNN.

Sense: Model Hardware Co-design for Accelerating Sparse CNN on Systolic Array

Systimator: A Design Space Exploration Methodology for Systolic Array based CNNs Acceleration on the FPGA-based Edge Nodes