Abstract:In order to process complex and large-scale graph data numerous distributed graph-parallel computing platforms have been proposed. However, excessive communications among computing nodes in these systems not only aggravate the network I/O workload of the underlying computing hardware systems but may also cause a decrease in runtime performance and scalability. In this paper, we propose and implement a system called Ligraph, which computes large-scale graph data in distributed mode with lightweight communication overhead. Ligraph is similar to PowerGraph system with three new features: (1) a Gather partial sum difference based computing model; (2) a corresponding lightweight Gather communication mechanism; (3) for PageRank-like algorithms Ligraph additionally employs a lightweight synchronizing communication mechanism and an edge direction-aware graph partition strategy proposed by our former work LightGraph, which is specially designed for PageRank-like algorithms. We have conducted extensive experiments using real-world data sets, and our results verified the effectiveness of Ligraph on reducing the communication overhead and improving the runtime performance and the scalability compared with PowerGraph and LightGraph. For example, compared with PowerGraph under Random partition scenario Ligraph can not only reduce up to 35.2 percent of the communication overhead but also cut up to 21.8 percent of the runtime for PageRank algorithm while processing Twitter data set. Our experiment results also demonstrate that compared with several other representative existing systems Ligraph also outperforms them in graph computing rate.

Lumping Algorithms for Computing Google’s PageRank and Its Derivative, with Attention to Unreferenced Nodes

On computing PageRank via lumping the Google matrix

Two Accelerated Non-backtracking PageRank Algorithms for Large-scale Networks

Enhanced preprocessed multi-step splitting iterations for computing PageRank

Weak dangling block reordering and multi-step block compression for efficiently computing and updating PageRank solutions

On computing HITS ExpertRank via lumping the hub matrix

Revisiting Local PageRank Estimation on Undirected Graphs: Simple and Optimal

Calculating webpage importance with site structure constraints

Efficient Algorithms for Personalized PageRank Computation: A Survey

N-Step Pagerank For Web Search

Distributed PageRank Computation Based on Iterative Aggregation-Disaggregation Methods.

Virtual Web Based Personalized PageRank Updating

A Power–Arnoldi Algorithm for Computing PageRank

Combination of in-memory graph computation with mapreduce: a subgraph-centric method of pagerank

Accelerating the Arnoldi-Type Algorithm for the PageRank Problem and the ProteinRank Problem

Randomized Incremental Algorithms For The Pagerank Computation

A Distributed Graph-Parallel Computing System with Lightweight Communication Overhead

Calibratable Hetero-NodeRank for measuring node influence

Multilevel Adaptive Aggregation for Markov Chains, with Application to Web Ranking

A Survey on Personalized PageRank Computation Algorithms

An Improved PageRank Algorithm Based on Web Content