An Efficient Framework For Incremental Graph Computation

Qiang Liu,Xiaoshe Dong,Heng Chen,Zheng Dong Zhu,Yinfeng Wang

DOI: https://doi.org/10.1109/TrustCom.2016.221

2016-01-01

Abstract:Graph computation has become increasingly popular in emerging applications such as social networks and web graphs. In practice, graph is typically large and frequently updated with small changes. However, traditional graph processing systems process the evolving graph in a batch manner. To accelerate these large-scale graph applications, this paper proposes an efficient message reuse framework for incremental graph computation, called IncTracker. IncTracker automatically detects the input data changes and only processes the "changed" vertices by employing an efficient, fine-grained reuse mechanism. By reusing the intermediate messages and results of previous runs, the proposed framework can efficiently avoid redundant message communication in iterations. We compare the framework with Pregel on several SNAP datasets. The results show that the speedup ranges from 1.10x to 2.26x and the network traffic is significantly reduced when the incremental changes from 0% to 20%.