IncGraph:Large-Scale Graph Processing System for Real-Time and Incremental Computing

SHEN Lin,XUE Jilong,QU Zhi,YANG Zhi,DAI Yafei

DOI: https://doi.org/10.3778/j.issn.1673-9418.1305031

2013-01-01

Abstract:With the increasing popularity of online social networks(OSNs),more and more related applications need a com putation framework for processing large social graphs in a real-time manner.However,the existing graph processing sys tems,such as Google's Pregel,cannot achieve real-time performance due to processing the graph in global and batch man ners.Therefore,this paper proposes a new graph computation model that restricts the computation to a local area when a node is updated.In essence,it transforms the traditional batch and global computations to a series of incremental and local computations.By this way,people achieve the goal of real-time computation,and avoid the overhead of duplicate computa tion on nodes that have not changed.Based on this new computation model,this paper designs IncGraph,a real-time large graph processing system.To achieve an efficient and scalable implementation,this paper uses graph partition techniques to exploit the computation locality in social graph,moreover,uses proactive computing trigger and inversing update chain to realize the real-time and reliable computing.Finally,this paper uses real social network data to validate the system performance.IncGraph can provide social network applications a more flexible computation framework.