Abstract:In order to process complex and large-scale graph data numerous distributed graph-parallel computing platforms have been proposed. However, excessive communications among computing nodes in these systems not only aggravate the network I/O workload of the underlying computing hardware systems but may also cause a decrease in runtime performance and scalability. In this paper, we propose and implement a system called Ligraph, which computes large-scale graph data in distributed mode with lightweight communication overhead. Ligraph is similar to PowerGraph system with three new features: (1) a Gather partial sum difference based computing model; (2) a corresponding lightweight Gather communication mechanism; (3) for PageRank-like algorithms Ligraph additionally employs a lightweight synchronizing communication mechanism and an edge direction-aware graph partition strategy proposed by our former work LightGraph, which is specially designed for PageRank-like algorithms. We have conducted extensive experiments using real-world data sets, and our results verified the effectiveness of Ligraph on reducing the communication overhead and improving the runtime performance and the scalability compared with PowerGraph and LightGraph. For example, compared with PowerGraph under Random partition scenario Ligraph can not only reduce up to 35.2 percent of the communication overhead but also cut up to 21.8 percent of the runtime for PageRank algorithm while processing Twitter data set. Our experiment results also demonstrate that compared with several other representative existing systems Ligraph also outperforms them in graph computing rate.

Scalable distributed Louvain algorithm for community detection in large graphs

GVE-Louvain: Fast Louvain Algorithm for Community Detection in Shared Memory Setting

An Approach for Addressing Internally-Disconnected Communities in Louvain Algorithm

DF Louvain: Fast Incrementally Expanding Approach for Community Detection on Dynamic Graphs

Memory-Efficient Community Detection on Large Graphs Using Weighted Sketches

An Improved Louvain Algorithm for Community Detection

GVE-Leiden: Fast Leiden Algorithm for Community Detection in Shared Memory Setting

A Scalable Community Detection Algorithm for Large Graphs Using Stochastic Block Models.

Selecting a suitable Parallel Label-propagation based algorithm for Disjoint Community Detection

Heuristic-based Dynamic Leiden Algorithm for Efficient Tracking of Communities on Evolving Graphs

MIRACLE: A Multiple Independent Random Walks Community Parallel Detection Algorithm for Big Graphs

A Distributed Graph-Parallel Computing System with Lightweight Communication Overhead

Distributed parallel deep learning with a hybrid backpropagation-particle swarm optimization for community detection in large complex networks

Distributed structural clustering on large graph

Fast unfolding of communities in large networks: 15 years later

Online Community Detection for Large Complex Networks.

Distributed Pseudo-Likelihood Method for Community Detection in Large-Scale Networks

Memory Efficient GPU-based Label Propagation Algorithm (LPA) for Community Detection on Large Graphs

A Survey of Distributed Graph Algorithms on Massive Graphs