Abstract:Graph neural network (GNN) is a promising emerging application for link prediction, recommendation, etc. Existing hardware innovation is limited to single-machine GNN (SM-GNN), however, the enterprises usually adopt huge graph with large-scale distributed GNN (LSD-GNN) that has to be carried out with distributed in-memory storage. The LSD-GNN is very different from SM-GNN in terms of system architecture demand, workflow and operators, and hence characterizations. In this paper, we first quantitively characterize the LSD-GNN with industrial-grade framework and application, summarize that its challenges lie in graph sampling, including distributed graph access, long latency, and underutilized communication and memory bandwidth. These challenges are missing from previous SM-GNN targeted researches. We then propose a customized hardware architecture to solve the challenges, including a fully pipelined access engine architecture for graph access and sampling, a low-latency and bandwidth-efficient customized memory-over-fabric hardware, and a RISC-V centric control system providing good programma-bility. We implement the proposed architecture with full software support in a 4-card FPGA heterogeneous proof-of-concept (PoC) system. Based on the measurement result from the FPGA PoC, we demonstrate a single FPGA can provide up to 894 vCPU's sampling capability. With the goal of being profitable, programmable, and scalable, we further integrate the architecture to FPGA cloud (FaaS) at hyperscale, along with the industrial software framework. We explicitly explore eight FaaS architectures that carry out the proposed accelerator hardware. We finally conclude that off-the-shelf FaaS.base can already provide 2.47× performance per dollar improvement with our hardware. With architecture optimizations, FaaS.comm-opt with customized FPGA fabrics pushes the benefit to 7.78×, and FaaS.mem-opt with FPGA local DRAM and high-speed links to GPU further unleash the benefit to 12.58×.

DF-GAS: a Distributed FPGA-as-a-Service Architecture Towards Billion-Scale Graph-based Approximate Nearest Neighbor Search.

Hyperscale FPGA-as-a-service Architecture for Large-Scale Distributed Graph Neural Network

FusionANNS: An Efficient CPU/GPU Cooperative Processing Architecture for Billion-scale Approximate Nearest Neighbor Search

A Real-Time Adaptive Multi-Stream GPU System for Online Approximate Nearest Neighborhood Search

Processing-In-Hierarchical-Memory Architecture for Billion-Scale Approximate Nearest Neighbor Search

NDSEARCH: Accelerating Graph-Traversal-Based Approximate Nearest Neighbor Search through Near Data Processing

BANG: Billion-Scale Approximate Nearest Neighbor Search using a Single GPU

CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs

Bridging Software-Hardware for CXL Memory Disaggregation in Billion-Scale Nearest Neighbor Search

Accelerating Large-Scale Graph-based Nearest Neighbor Search on a Computational Storage Platform

Scalable Billion-point Approximate Nearest Neighbor Search Using SmartSSDs.

Co-design Hardware and Algorithm for Vector Search

Fast Approximate Nearest Neighbor Search with the Navigating Spreading-out Graph.

L-FNNG: Accelerating Large-Scale KNN Graph Construction on CPU-FPGA Heterogeneous Platform

Proxima: Near-storage Acceleration for Graph-based Approximate Nearest Neighbor Search in 3D NAND

FNNG: A High-Performance FPGA-based Accelerator for K-Nearest Neighbor Graph Construction.

GPU-accelerated Proximity Graph Approximate Nearest Neighbor Search and Construction

SA-GNAS: Seed Architecture Expansion for Efficient Large-scale Graph Neural Architecture Search

Efficient and Effective Retrieval of Dense-Sparse Hybrid Vectors using Graph-based Approximate Nearest Neighbor Search

Optimizing Graph-based Approximate Nearest Neighbor Search: Stronger and Smarter

DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inference