Abstract:Spiking Neural Networks (SNNs) offer a promising alternative to Artificial Neural Networks (ANNs) for deep learning applications, particularly in resource-constrained systems. This is largely due to their inherent sparsity, influenced by factors such as the input dataset, the length of the spike train, and the network topology. While a few prior works have demonstrated the advantages of incorporating sparsity into the hardware design, especially in terms of reducing energy consumption, the impact on hardware resources has not yet been explored. This is where design space exploration (DSE) becomes crucial, as it allows for the optimization of hardware performance by tailoring both the hardware and model parameters to suit specific application needs. However, DSE can be extremely challenging given the potentially large design space and the interplay of hardware architecture design choices and application-specific model parameters. In this paper, we propose a flexible hardware design that leverages the sparsity of SNNs to identify highly efficient, application-specific accelerator designs. We develop a high-level, cycle-accurate simulation framework for this hardware and demonstrate the framework's benefits in enabling detailed and fine-grained exploration of SNN design choices, such as the layer-wise logical-to-hardware ratio (LHR). Our experimental results show that our design can (i) achieve up to $76\%$ reduction in hardware resources and (ii) deliver a speed increase of up to $31.25\times$, while requiring $27\%$ fewer hardware resources compared to sparsity-oblivious designs. We further showcase the robustness of our framework by varying spike train lengths with different neuron population sizes to find the optimal trade-off points between accuracy and hardware latency.

Analyzing the Design Space of Spatial Tensor Accelerators on FPGAs

TensorLib - A Spatial Accelerator Generation Framework for Tensor Algebra.

Design Space Exploration of FPGA-based Accelerators with Multi-Level Parallelism

An accelerator-aware microarchitecture simulator for design space exploration

Automatic Generation of Spatial Accelerator for Tensor Algebra

Software-defined Design Space Exploration for an Efficient DNN Accelerator Architecture

EMS: Efficient Memory Subsystem Synthesis for Spatial Accelerators

Design and Application Space Exploration of a Domain-Specific Accelerator System

AERO: Design Space Exploration Framework for Resource-Constrained CNN Mapping on Tile-Based Accelerators

Evaluating Spatial Accelerator Architectures with Tiled Matrix-Matrix Multiplication

Evaluation Mappings of Spatial Accelerator Based On Data Placement

AutoDSE: Enabling Software Programmers to Design Efficient FPGA Accelerators

Serving Multi-DNN Workloads on FPGAs: A Coordinated Architecture, Scheduling, and Mapping Perspective.

A Comprehensive Evaluation of FPGA-Based Spatial Acceleration of LLMs

Optimized Spatial Architecture Mapping Flow for Transformer Accelerators

Spatial: a language and compiler for application accelerators

SSR: Spatial Sequential Hybrid Architecture for Latency Throughput Tradeoff in Transformer Acceleration

Efficient Processing of Sparse Tensor Decomposition via Unified Abstraction and PE-Interactive Architecture

Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search

HASCO: Towards Agile HArdware and Software CO-design for Tensor Computation

Design Space Exploration of Sparsity-Aware Application-Specific Spiking Neural Network Accelerators