GraphScope Flex: LEGO-like Graph Computing Stack
Tao He,Shuxian Hu,Longbin Lai,Dongze Li,Neng Li,Xue Li,Lexiao Liu,Xiaojian Luo,Binqing Lyu,Ke Meng,Sijie Shen,Li Su,Lei Wang,Jingbo Xu,Wenyuan Yu,Weibin Zeng,Lei Zhang,Siyuan Zhang,Jingren Zhou,Xiaoli Zhou,Diwen Zhu
2023-12-19
Abstract:Graph computing has become increasingly crucial in processing large-scale graph data, with numerous systems developed for this purpose. Two years ago, we introduced GraphScope as a system addressing a wide array of graph computing needs, including graph traversal, analytics, and learning in one system. Since its inception, GraphScope has achieved significant technological advancements and gained widespread adoption across various industries. However, one key lesson from this journey has been understanding the limitations of a "one-size-fits-all" approach, especially when dealing with the diversity of programming interfaces, applications, and data storage formats in graph computing. In response to these challenges, we present GraphScope Flex, the next iteration of GraphScope. GraphScope Flex is designed to be both resource-efficient and cost-effective, while also providing flexibility and user-friendliness through its LEGO-like modularity. This paper explores the architectural innovations and fundamental design principles of GraphScope Flex, all of which are direct outcomes of the lessons learned during our ongoing development process. We validate the adaptability and efficiency of GraphScope Flex with extensive evaluations on synthetic and real-world datasets. The results show that GraphScope Flex achieves 2.4X throughput and up to 55.7X speedup over other systems on the LDBC Social Network and Graphalytics benchmarks, respectively. Furthermore, GraphScope Flex accomplishes up to a 2,400X performance gain in real-world applications, demonstrating its proficiency across a wide range of graph computing scenarios with increased effectiveness.
Distributed, Parallel, and Cluster Computing,Databases