Abstract:In order to process complex and large-scale graph data numerous distributed graph-parallel computing platforms have been proposed. However, excessive communications among computing nodes in these systems not only aggravate the network I/O workload of the underlying computing hardware systems but may also cause a decrease in runtime performance and scalability. In this paper, we propose and implement a system called Ligraph, which computes large-scale graph data in distributed mode with lightweight communication overhead. Ligraph is similar to PowerGraph system with three new features: (1) a Gather partial sum difference based computing model; (2) a corresponding lightweight Gather communication mechanism; (3) for PageRank-like algorithms Ligraph additionally employs a lightweight synchronizing communication mechanism and an edge direction-aware graph partition strategy proposed by our former work LightGraph, which is specially designed for PageRank-like algorithms. We have conducted extensive experiments using real-world data sets, and our results verified the effectiveness of Ligraph on reducing the communication overhead and improving the runtime performance and the scalability compared with PowerGraph and LightGraph. For example, compared with PowerGraph under Random partition scenario Ligraph can not only reduce up to 35.2 percent of the communication overhead but also cut up to 21.8 percent of the runtime for PageRank algorithm while processing Twitter data set. Our experiment results also demonstrate that compared with several other representative existing systems Ligraph also outperforms them in graph computing rate.

Survey on computing models in graph data analysis systems

Large-scale graph processing systems: a survey

Graph Computing Systems for Large Scale Graph Analysis

Reviewing Large Graph Computing from a System Perspective

The Survey of Complex Dynamical Networks Models

Survey on Technologies of Distributed Graph Processing Systems

Survey on Dynamic Graph Pattern Matching Technologies

Research on Modeling of Complex System Based on Graph Theory

Large scale graph processing systems: survey and an experimental evaluation

Software Systems Implementation and Domain-Specific Architectures Towards Graph Analytics

Quantitative Analysis of Graph Algorithms: Models and Optimization Methods

The Scale-Free Feature and Evolving Model of Large-Scale Software Systems

Survey of External Memory Large-Scale Graph Processing on a Multi-Core System

A Survey of Distributed Graph Algorithms on Massive Graphs

A Distributed Graph-Parallel Computing System with Lightweight Communication Overhead

A Survey on Mapping Semi-Structured Data and Graph Data to Relational Data

Graph Stream: Model, Algorithm and System

Big Graph Management Based on Scalable Computing Platforms

Models of parallel computation: a survey and classification