Abstract:The unprecedented accuracy of convolutional neural networks (CNNs) across a broad range of AI tasks has led to their widespread deployment in mobile and embedded settings. In a pursuit for high-performance and energy-efficient inference, significant research effort has been invested in the design of FPGA-based CNN accelerators. In this context, single computation engines constitute a popular approach to support diverse CNN modes without the overhead of fabric reconfiguration. Nevertheless, this flexibility often comes with significantly degraded performance on memory-bound layers and resource underutilisation due to the suboptimal mapping of certain layers on the engine's fixed configuration. In this work, we investigate the implications in terms of CNN engine design for a class of models that introduce a pre-convolution stage to decompress the weights at run time. We refer to these approaches as on-the-fly. This paper presents unzipFPGA, a novel CNN inference system that counteracts the limitations of existing CNN engines. The proposed framework comprises a novel CNN hardware architecture that introduces a weights generator module that enables the on-chip on-the-fly generation of weights, alleviating the negative impact of limited bandwidth on memory-bound layers. We further enhance unzipFPGA with an automated hardware-aware methodology that tailors the weights generation mechanism to the target CNN-device pair, leading to an improved accuracy-performance balance. Finally, we introduce an input selective processing element (PE) design that balances the load between PEs in suboptimally mapped layers. The proposed framework yields hardware designs that achieve an average of 2.57x performance efficiency gain over highly optimised GPU designs for the same power constraints and up to 3.94x higher performance density over a diverse range of state-of-the-art FPGA-based CNN accelerators.

Flip-and-Patch: A fault-tolerant technique for on-chip memories of CNN accelerators at low supply voltage

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Shavette: Low Power Neural Network Acceleration via Algorithm-level Error Detection and Undervolting

An Experimental Study of Reduced-Voltage Operation in Modern FPGAs for Neural Network Acceleration

NeuralFuse: Learning to Recover the Accuracy of Access-Limited Neural Network Inference in Low-Voltage Regimes

Evaluating Built-in ECC of FPGA on-chip Memories for the Mitigation of Undervolting Faults

Implementation of Highly Reliable Convolutional Neural Network with Low Overhead on Field-Programmable Gate Array

A Low-Latency DNN Accelerator Enabled by DFT-Based Convolution Execution Within Crossbar Arrays

An Energy-Efficient Neural Network Accelerator With Improved Resilience Against Fault Attacks

Detect and Replace: Efficient Soft Error Protection of FPGA-Based CNN Accelerators

Efficient Error-Tolerant Quantized Neural Network Accelerators

Drop-Connect as a Fault-Tolerance Approach for RRAM-based Deep Neural Network Accelerators

SoftSNN: Low-Cost Fault Tolerance for Spiking Neural Network Accelerators under Soft Errors

Algorithm-Based Fault Tolerance for Convolutional Neural Networks

Towards Reliable AI Applications Via Algorithm-Based Fault Tolerance on NVDLA

Mitigating Memory Wall Effects in CNN Engines with On-the-Fly Weights Generation

Reliable and Energy Efficient MLC STT-RAM Buffer for CNN Accelerators

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

An Adaptive Read Control Voltage Scheme for Reliability Enhancement of Flash-Based In-Memory Computing Architecture for Neural Network

Cost-Effective Fault Tolerance for CNNs Using Parameter Vulnerability Based Hardening and Pruning

RescueSNN: Enabling Reliable Executions on Spiking Neural Network Accelerators under Permanent Faults