Abstract:In a wide variety of emerging data-intensive applications, such as social network analysis, Web document clustering, entity resolution, and detection of consistently co-expressed genes in systems biology, the detection of dense subgraphs (cliques and approximate or quasi-cliques) is an essential component. Unfortunately, these problems are NP-Complete and thus computationally intensive at scale — hence there is a need to come up with techniques for distributing the computation across multiple machines such that the computation, which is too time-consuming on a single machine, can be efficiently performed on a machine cluster given that it is large enough. In this paper, we first propose a new approach for maximal clique and quasi-clique enumeration, which identifies dense subgraphs by recursive graph partitioning. Given a connected graph G = (V,E), it has a space complexity of O(|E|) and a time complexity of O(|E|μ(G)), where μ(G) represents the number of different cliques (quasi-cliques) existing in G. It recursively divides a graph until each task is sufficiently small to be processed in parallel. We then develop parallel solutions and demonstrate how graph partitioning can enable effective load balancing. Finally, we evaluate the performance of the proposed approach on real and synthetic graph data and show that it performs considerably better than existing approaches in both centralized and parallel settings. In the parallel setting, it can achieve the speedups of up to 10x over existing approaches on large graphs. Our parallel algorithms are implemented and evaluated on MapReduce, a popular shared-nothing parallel framework, but can easily generalize to other shared-nothing or shared-memory parallel frameworks.

Study on Resource Efficiency of Distributed Graph Processing

Evaluation and Analysis of Distributed Graph-Parallel Processing Frameworks

Experimental Analysis of Distributed Graph Systems

A Distributed Graph-Parallel Computing System with Lightweight Communication Overhead

Survey on Technologies of Distributed Graph Processing Systems

Distributed Graph Processing - Techniques and Systems.

Graph3S: A Simple, Speedy and Scalable Distributed Graph Processing System

A Survey of Distributed Graph Algorithms on Massive Graphs

Parallelizing Clique and Quasi-Clique Detection over Graph Data

Energy Efficiency of Large Scale Graph Processing Platforms

Towards Efficient Graph Processing in Geo-Distributed Data Centers

High-Level Programming Abstractions for Distributed Graph Processing

Distributed PathGraph: A Cluster Centric Framework for Distributed Processing Graph

Large-scale graph processing systems: a survey

A Comprehensive Study for Essentiality of Graph Based Distributed SPARQL Query Processing.

Efficient And Scalable Mining Of Frequent Subgraphs Using Distributed Graph Processing Systems

Distributed Graph Algorithms: From Local Data to Global Solutions

Evaluation and Trade-offs of Graph Processing for Cloud Services

Distributed Graph Pattern Matching