ReGraph: A Graph Processing Framework that Alternately Shrinks and Repartitions the Graph.

Xue Li,Mingxing Zhang,Kang Chen,Yongwei Wu

DOI: https://doi.org/10.1145/3205289.3205292

2018-01-01

Abstract:"Think Like a Sub-Graph (TLASG)" is a philosophy proposed for guiding the design of graph-oriented programming models. As TLASG-based models allow information to flow freely inside a partition, they usually require much fewer iterations to converge when compared with "Think Like a Vertex (TLAV)"-based models. In this paper, we further explore the idea of TLASG by enabling users to 1) proactively repartition the graph; and 2) efficiently scale down the problem's size. With these methods, our novel TLASG-based distributed graph processing system ReGraph requires even fewer iterations (typically ≤ 6) to converge, and hence achieves better performance (up to 45.4X) and scalability than existing TLAV and TLASG-based frameworks. Moreover, we show that these optimizations can be enabled without a large change in the programming model. We also implement our novel algorithm on top of Spark directly and compare it with other Spark-based implementation, which shows that our speedup is not bounded to our own platform.