Abstract:FPGAs are increasingly popular in cloud environments for their ability to offer on-demand acceleration and improved compute efficiency. Providers would like to increase utilization, by multiplexing customers on a single device, similar to how processing cores and memory are shared. Nonetheless, multi-tenancy still faces major architectural limitations including: a) inefficient sharing of memory interfaces across hardware tasks exacerbated by technological limitations and peculiarities, b) insufficient solutions for performance and data isolation and high quality of service, c) absent or simplistic allocation strategies to effectively distribute external FPGA memory across hardware tasks. This paper presents a full-stack solution for enabling multi-tenancy on FPGAs. Specifically, our work proposes an intra-fpga virtualization layer to share FPGA interfaces and its resources across tenants. To achieve efficient inter-connectivity between virtual FPGAs (vFGPAs) and external interfaces, we employ a compact network-on-chip architecture to optimize resource utilization. Dedicated memory management units implement the concept of virtual memory in FPGAs, providing mechanisms to isolate the address space and enable memory protection. We also introduce a memory segmentation scheme to effectively allocate FPGA address space and enhance isolation through hardware-software support, while preserving the efficacy of memory transactions. We assess our solution on an Alveo U250 Data Center FPGA Card, employing ten real-world benchmarks from the Rodinia and Rosetta suites. Our framework preserves the performance of hardware tasks from a non-virtualized environment, while enhancing the device aggregate throughput through resource sharing; up to 3.96x in isolated and up to 2.31x in highly congested settings, where an external interface is shared across four vFPGAs. Finally, our work ensures high-quality of service, with hardware tasks achieving up to 0.95x of their native performance, even when resource sharing introduces interference from other accelerators.

Enabling Elastic Resource Management in Cloud FPGAs Via A Multi-layer Collaborative Approach.

Engaging Heterogeneous FPGAs in the Cloud.

Towards Hardware Support for FPGA Resource Elasticity

Enabling FPGAs in the cloud.

FPGA Resource Pooling in Cloud Computing

Architectural support for sharing, isolating and virtualizing FPGA resources.

Deploying Multi-tenant FPGAs within Linux-based Cloud Infrastructure

Online scheduling for FPGA computation in the Cloud

Migration of long-running Tasks between Reconfigurable Resources using Virtualization

Evaluation of Static Mapping for Dynamic Space-Shared Multi-task Processing on FPGAs

GLRM: Geometric Layout-Based Resource Management Method on Multiple Field Programmable Gate Array Systems

Enabling Efficient and Flexible FPGA Virtualization for Deep Learning in the Cloud

A Study of FPGA Virtualization and Accelerator Scheduling

RC3E: Provision and Management of Reconfigurable Hardware Accelerators in a Cloud Environment

Enabling OpenMP Task Parallelism on Multi-FPGAs

FOS: A Modular FPGA Operating System for Dynamic Workloads

Minimize the Make-span of Batched Requests for FPGA Pooling in Cloud Computing

Resource Partitioning and Application Scheduling with Module Merging on Dynamically and Partially Reconfigurable FPGAs

Power Aware Scheduling of Tasks on FPGAs in Data Centers

Workload-Aware Opportunistic Energy Efficiency in Multi-FPGA Platforms

TAPA-CS: Enabling Scalable Accelerator Design on Distributed HBM-FPGAs