Abstract:Graph neural network (GNN) is a promising emerging application for link prediction, recommendation, etc. Existing hardware innovation is limited to single-machine GNN (SM-GNN), however, the enterprises usually adopt huge graph with large-scale distributed GNN (LSD-GNN) that has to be carried out with distributed in-memory storage. The LSD-GNN is very different from SM-GNN in terms of system architecture demand, workflow and operators, and hence characterizations. In this paper, we first quantitively characterize the LSD-GNN with industrial-grade framework and application, summarize that its challenges lie in graph sampling, including distributed graph access, long latency, and underutilized communication and memory bandwidth. These challenges are missing from previous SM-GNN targeted researches. We then propose a customized hardware architecture to solve the challenges, including a fully pipelined access engine architecture for graph access and sampling, a low-latency and bandwidth-efficient customized memory-over-fabric hardware, and a RISC-V centric control system providing good programma-bility. We implement the proposed architecture with full software support in a 4-card FPGA heterogeneous proof-of-concept (PoC) system. Based on the measurement result from the FPGA PoC, we demonstrate a single FPGA can provide up to 894 vCPU's sampling capability. With the goal of being profitable, programmable, and scalable, we further integrate the architecture to FPGA cloud (FaaS) at hyperscale, along with the industrial software framework. We explicitly explore eight FaaS architectures that carry out the proposed accelerator hardware. We finally conclude that off-the-shelf FaaS.base can already provide 2.47× performance per dollar improvement with our hardware. With architecture optimizations, FaaS.comm-opt with customized FPGA fabrics pushes the benefit to 7.78×, and FaaS.mem-opt with FPGA local DRAM and high-speed links to GPU further unleash the benefit to 12.58×.

FpgaNIC: An FPGA-based Versatile 100Gb SmartNIC for GPUs

Efficient PC-FPGA Communication over Gigabit Ethernet

A 400Gbit Ethernet core enabling High Data Rate Streaming from FPGAs to Servers and GPUs in Radio Astronomy

Extracting TCPIP Headers at High Speed for the Anonymized Network Traffic Graph Challenge

FPGA-Based AI Smart NICs for Scalable Distributed AI Training Systems

NetStorageFPGA—A prototyping platform for building high-performance transmission and storage systems using Field Programmable Gate Array (FPGA) hardware

High-Performance Reconfigurable Pipeline Implementation for FPGA-Based SmartNIC

G-NET: Effective GPU Sharing in NFV Systems.

An Offloading Architecture of Lossless Compression Based on Smart NIC

Hyperscale FPGA-as-a-service Architecture for Large-Scale Distributed Graph Neural Network

Leveraging Silicon-Photonic Noc For Designing Scalable Gpus

NPGPU: Network processing on graphics processing units

HPIPE: Heterogeneous Layer-Pipelined and Sparse-Aware CNN Inference for FPGAs

FPX-NIC: An FPGA-Accelerated 4K Ultra-High-Definition Neural Video Coding System

ClickNP: Highly flexible and High-performance Network Processing with Reconfigurable Hardware.

A Survey of FPGA-Based Neural Network Accelerator

Gpu-Accelerated Evaluation Platform For High Fidelity Network Modeling

Nuclei: GPU-Accelerated Many-Core Network Coding

Accelerating Mobile Applications at the Network Edge with Software-Programmable FPGAs.

A High-Performance and Flexible Architecture for Accelerating SDN on the MPSoC Platform

A Ubiquitous Machine Learning Accelerator With Automatic Parallelization on FPGA