Abstract:Graph analytics is an important way to understand relationships in real-world applications. At the age of big data, graphs have grown to billions of edges. This motivates distributed graph processing. Graph processing frameworks ask programmers to specify graph computations in user- defined functions (UDFs) of graph-oriented programming model. Due to the nature of distributed execution, current frameworks cannot precisely enforce the semantics of UDFs, leading to unnecessary computation and communication. In essence, there exists a gap between programming model and runtime execution. This paper proposes SympleGraph, a novel distributed graph processing framework that precisely enforces loop-carried dependency, i.e., when a condition is satisfied by a neighbor, all following neighbors can be skipped. SympleGraph instruments the UDFs to express the loop-carried dependency, then the distributed execution framework enforces the precise semantics by performing dependency propagation dynamically. Enforcing loop-carried dependency requires the sequential processing of the neighbors of each vertex distributed in different nodes. Therefore, the major challenge is to enable sufficient parallelism to achieve high performance. We propose to use circulant scheduling in the framework to allow different machines to process disjoint sets of edges/vertices in parallel while satisfying the sequential requirement. It achieves a good trade-off between precise semantics and parallelism. The significant speedups in most graphs and algorithms indicate that the benefits of eliminating unnecessary computation and communication overshadow the reduced parallelism. Communication efficiency is further optimized by 1) selectively propagating dependency for large-degree vertices to increase net benefits; 2) double buffering to hide communication latency. In a 16-node cluster, SympleGraph outperforms the state-of-the-art system Gemini and D-Galois on average by 1.42× and 3.30×, and up to 2.30× and 7.76×, respectively. The communication reduction compared to Gemini is 40.95% on average and up to 67.48%.

Graphu: A Unified Vertex-Centric Parallel Graph Processing Platform

DFA-G: A Unified Programming Model for Vertex-Centric Parallel Graph Processing

GraphHP: A Hybrid Platform for Iterative Graph Processing.

A Distributed Graph-Parallel Computing System with Lightweight Communication Overhead

Evaluation and Analysis of Distributed Graph-Parallel Processing Frameworks

GGPA: A General Graph Processing Architecture with Flexible Execution Paradigm

SympleGraph: Distributed Graph Processing with Precise Loop-Carried Dependency Guarantee.

GraphM

FBSGraph: Accelerating Asynchronous Graph Processing Via Forward and Backward Sweeping

Graph3S: A Simple, Speedy and Scalable Distributed Graph Processing System

Big Graph Management Based on Scalable Computing Platforms

Garaph: Efficient GPU-accelerated Graph Processing on a Single Machine with Balanced Replication.

Domino: an Asynchronous and Energy-efficient Accelerator for Graph Processing: (abstract Only)

Streamlined Asynchronous Graph Processing Framework

Processing Concurrent Graph Analytics with Decoupled Computation Model.

An Efficient Iterative Graph Data Processing Framework Based on Bulk Synchronous Parallel Model

Fargraph+: Excavating the Parallelism of Graph Processing Workload on RDMA-based Far Memory System

RGraph: Asynchronous Graph Processing Based on Asymmetry of Remote Direct Memory Access

High Performance Graph Analytics with Productivity on Hybrid CPU-GPU Platforms

Graph Analytics Through Fine-Grained Parallelism.

A Vertex-Centric Graph Simulation Algorithm for Large Graphs.