Abstract:Recent advancements in large language models (LLMs) boasting billions of parameters have generated a significant demand for efficient deployment in inference workloads. The majority of existing approaches rely on temporal architectures that reuse hardware units for different network layers and operators. However, these methods often encounter challenges in achieving low latency due to considerable memory access overhead. This paper investigates the feasibility and potential of model-specific spatial acceleration for LLM inference on FPGAs. Our approach involves the specialization of distinct hardware units for specific operators or layers, facilitating direct communication between them through a dataflow architecture while minimizing off-chip memory accesses. We introduce a comprehensive analytical model for estimating the performance of a spatial LLM accelerator, taking into account the on-chip compute and memory resources available on an FPGA. Through our analysis, we can determine the scenarios in which FPGA-based spatial acceleration can outperform its GPU-based counterpart. To enable more productive implementations of an LLM model on FPGAs, we further provide a library of high-level synthesis (HLS) kernels that are composable and reusable. This library will be made available as open-source. To validate the effectiveness of both our analytical model and HLS library, we have implemented BERT and GPT2 on an AMD Alveo U280 FPGA device. Experimental results demonstrate our approach can achieve up to 13.4x speedup when compared to previous FPGA-based accelerators for the BERT model. For GPT generative inference, we attain a 2.2x speedup compared to DFX, an FPGA overlay, in the prefill stage, while achieving a 1.9x speedup and a 5.7x improvement in energy efficiency compared to the NVIDIA A100 GPU in the decode stage.

LLM for Complex Signal Processing in FPGA-based Software Defined Radios: A Case Study on FFT

The Power of Large Language Models for Wireless Communication System Development: A Case Study on FPGA Platforms

Design and implementation of LMS adaptive filter algorithm based on FPGA

FPGA Deployment of LFADS for Real-time Neuroscience Experiments

The Design and Hardware Implementation of OFDM System Receiver Based on FPGA

Accelerating FPGA-Based Wi-Fi Transceiver Design and Prototyping by High-Level Synthesis

Implementation of Deep Learning-based Automatic Modulation Classifier on FPGA SDR Platform

A Comprehensive Evaluation of FPGA-Based Spatial Acceleration of LLMs

An Automated FPGA-based Framework for Rapid Prototyping of Nonbinary LDPC Codes

LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation

Design and Implementation of Efficient FFT Processor for Multicarrier System

Large Language Model Enabled Semantic Communication Systems

Digital ASIC Design with Ongoing LLMs: Strategies and Prospects

From English to ASIC: Hardware Implementation with Large Language Model

Designing Efficient LLM Accelerators for Edge Devices

EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Models

C2HLSC: Leveraging Large Language Models to Bridge the Software-to-Hardware Design Gap

Research and Implementation of Soft-Decision Demodulation for 8PSK Based on FPGA

Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference

Evaluating LLMs for Hardware Design and Test

FPGA Implementation of LSTM Based on Automatic Speech Recognition