RisGraph: A Real-Time Streaming System for Evolving Graphs to Support Sub-millisecond Per-update Analysis at Millions Ops/s

Guanyu Feng,Zixuan Ma,Daixuan Li,Shengqi Chen,Xiaowei Zhu,Wentao Han,Wenguang Chen

DOI: https://doi.org/10.1145/3448016.3457263

2021-01-01

Abstract:Evolving graphs in the real world are large-scale and constantly changing, ashundreds of thousands of updates may come every second. Monotonic algorithmssuch as Reachability and Shortest Path are widely used in real-time analyticsto gain both static and temporal insights and can be accelerated by incrementalcomputing. Existing streaming systems adopt the incremental computing model andachieve either low latency or high throughput, but not both. However, both highthroughput and low latency are required in real scenarios such as financialfraud detection. This paper presents RisGraph, a real-time streaming systemthat provides low-latency analysis for each update with high throughput.RisGraph addresses the challenge with localized data access and inter-updateparallelism. We propose a data structure named Indexed Adjacency Lists and usesparse arrays and Hybrid Parallel Mode to enable localized data access. Toachieve inter-update parallelism, we propose a domain-specific concurrencycontrol mechanism based on the classification of safe and unsafe updates.Experiments show that RisGraph can ingest millions of updates per second forgraphs with several hundred million vertices and billions of edges, and theP999 processing time latency is within 20 milliseconds. RisGraph achievesorders-of-magnitude improvement on throughput when analyses are executed foreach update without batching and performs better than existing systems withbatches of up to 20 million updates.