Abstract: Graph neural networks (GNNs) have been demonstrated as a powerful tool for analyzing non-Euclidean graph data. However, the lack of efficient distributed graph learning systems severely hinders applications of GNNs, especially when graphs are big and GNNs are relatively deep. Herein, we present GraphTheta, the first distributed and scalable graph learning system built upon vertex-centric distributed graph processing with neural network operators implemented as user-defined functions. This system supports multiple training strategies and enables efficient and scalable big-graph learning on distributed (virtual) machines with low memory. To facilitate graph convolutions, GraphTheta puts forward a new graph learning abstraction named NN-TGAR to bridge the gap between graph processing and graph deep learning. A distributed graph engine is proposed to conduct the stochastic gradient descent optimization with a hybrid-parallel execution, and a new cluster-batched training strategy is supported. We evaluate GraphTheta using several datasets with network sizes ranging from small-, modest- to large-scale. Experimental results show that GraphTheta can scale well to 1,024 workers for training an in-house developed GNN on an industry-scale Alipay dataset of 1.4 billion nodes and 4.1 billion attributed edges, with a cluster of CPU virtual machines (dockers) of small memory each (5$\sim$12GB). Moreover, GraphTheta can outperform DistDGL by up to $2.02\times$, with better scalability, and GraphLearn by up to $30.56\times$. As for model accuracy, GraphTheta is capable of learning as good GNNs as existing frameworks. To the best of our knowledge, this work presents the largest edge-attributed GNN learning task in the literature.

WiseGraph

WiseGraph: Optimizing GNN with Joint Workload Partition of Graph and Operations.

ByteGNN: Efficient Graph Neural Network Training at Large Scale

Scalable and Efficient Full-Graph GNN Training for Large Graphs

Wide Graph Neural Network

Optimizing Task Placement and Online Scheduling for Distributed GNN Training Acceleration

Towards Efficient Large-Scale Graph Neural Network Computing.

Graph Neural Network Training with Data Tiering

Bipartite Graph Neural Networks for Efficient Node Representation Learning

Graph Neural Networks Inspired by Classical Iterative Algorithms

Ugrapher: High-Performance Graph Operator Computation Via Unified Abstraction for Graph Neural Networks.

GNN at the Edge: Cost-Efficient Graph Neural Network Processing over Distributed Edge Servers

Sparse Decomposition of Graph Neural Networks

Understanding and Bridging the Gaps in Current GNN Performance Optimizations

Graph Attention Multi-Layer Perceptron

GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy

AdaptGear: Accelerating GNN Training Via Adaptive Subgraph-Level Kernels on GPUs

Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective

Efficient Graph Neural Network Inference at Large Scale

Uplifting the Expressive Power of Graph Neural Networks through Graph Partitioning

EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks