Abstract:This work focuses on an efficient Agile design methodology for domain-specific accelerators. We employ feature-by-feature enhancement of a vertical development stack and apply it to the TVM/VTA inference accelerator. We have enhanced the VTA design space and enabled end-to-end support for additional workloads. This has been accomplished by augmenting the VTA micro-architecture and instruction set architecture (ISA), as well as by enhancing the TVM compilation stack to support a wide range of VTA configs. The VTA tsim implementation (CHISEL-based) has been enhanced with fully pipelined versions of the ALU/GEMM execution units. In tsim, memory width can now range between 8-64 bytes. Field widths have been made more flexible to support larger scratchpads. New instructions have been added: element-wise 8-bit multiplication to support depthwise convolution, and load with a choice of pad values to support max pooling. Support for more layers and better double buffering has also been added. Fully pipelining ALU/GEMM helps significantly: 4.9x fewer cycles with minimal area change to run ResNet-18 under the default config. Configs featuring a further 11.5x decrease in cycle count at a cost of 12x greater area can be instantiated. Many points on the area-performance pareto curve are shown, showcasing the balance of execution unit sizing, memory interface width, and scratchpad sizing. Finally, VTA is now able to run Mobilenet 1.0 and all layers for ResNets, including the previously disabled pooling and fully connected layers. The TVM/VTA architecture has always featured end-to-end workload evaluation on RTL in minutes. With our modifications, it now offers a much greater number of feasible configurations with a wide range of cost vs. performance. All capabilities mentioned are available in opensource forks while a subset of these capabilities have already been upstreamed.

TVB C++: A Fast and Flexible Back-End for The Virtual Brain

Vast TVB parameter space exploration: A Modular Framework for Accelerating the Multi-Scale Simulation of Human Brain Dynamics

Brain simulation as a cloud service: The Virtual Brain on EBRAINS

TVM: An Automated End-to-End Optimizing Compiler for Deep Learning

A Highly Configurable Hardware/Software Stack for DNN Inference Acceleration

A Novel Fully Hardware-Implemented SVD Solver Based on Ultra-Parallel BCV Jacobi Algorithm

Autotuning Apache TVM-based Scientific Applications Using Bayesian Optimization

Parallel Brain Simulator: A Multi-scale and Parallel Brain-Inspired Neural Network Modeling and Simulation Platform

Principles and Operation of Virtual Brain Twins

Efficient BVH-based Collision Detection Scheme with Ordering and Restructuring.

A Bandwidth Enhancement Method of VTA Based on Paralleled Memory Access Design

A Hardware-Software Blueprint for Flexible Deep Learning Specialization

fVDB : A Deep-Learning Framework for Sparse, Large Scale, and High Performance Spatial Intelligence

TVM: A Tile-based Video Management Framework

The Virtual Block Interface: A Flexible Alternative to the Conventional Virtual Memory Framework

fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence

BrainPy, a Flexible, Integrative, Efficient, and Extensible Framework for General-Purpose Brain Dynamics Programming.

BrainPy: a Flexible, Integrative, Efficient, and Extensible Framework Towards General-Purpose Brain Dynamics Programming

Digital Twin Brain: a simulation and assimilation platform for whole human brain

Aesptv: an Adaptive and Efficient Framework for Sparse Tensor-Vector Product Kernel on a High-Performance Computing Platform

Vc: A C++ library for explicit vectorization