Abstract:Recent work has initiated the study of dense graph processing using graph sketching methods, which drastically reduce space costs by lossily compressing information about the input graph. In this paper, we explore the strange and surprising performance landscape of sketching algorithms. We highlight both their surprising advantages for processing dense graphs that were previously prohibitively expensive to study, as well as the current limitations of the technique. Most notably, we show how sketching can avoid bottlenecks that limit conventional graph processing methods. Single-machine streaming graph processing systems are typically bottlenecked by CPU performance, and distributed graph processing systems are typically bottlenecked by network latency. We present Landscape, a distributed graph-stream processing system that uses linear sketching to distribute the CPU work of computing graph properties to distributed workers with no need for worker-to-worker communication. As a result, it overcomes the CPU and network bottlenecks that limit other systems. In fact, for the connected components problem, Landscape achieves a stream ingestion rate one-fourth that of maximum sustained RAM bandwidth, and is four times faster than random access RAM bandwidth. Additionally, we prove that for any sequence of graph updates and queries Landscape consumes at most a constant factor more network bandwidth than is required to receive the input stream. We show that this system can ingest up to 332 million stream updates per second on a graph with $2^{17}$ vertices. We show that it scales well with more distributed compute power: given a cluster of 40 distributed worker machines, it can ingest updates 35 times as fast as with 1 distributed worker machine. Landscape uses heuristics to reduce its query latency by up to four orders of magnitude over the prior state of the art.

Toward Continuous Pattern Detection over Evolving Large Graph with Snapshot Isolation

Continuous Pattern Detection over Billion-Edge Graph Using Distributed Framework

Enabling Window-Based Monotonic Graph Analytics with Reusable Transitional Results for Pattern-Consistent Queries

Evaluating Continuous Basic Graph Patterns over Dynamic Link Data Graphs

A Selectivity based approach to Continuous Pattern Detection in Streaming Graphs

RisGraph: A Real-Time Streaming System for Evolving Graphs to Support Sub-millisecond Per-update Analysis at Millions Ops/s

DGSVis: Visual Analysis of Hierarchical Snapshots in Dynamic Graph

Path-based Holistic Detection Plan for Multiple Patterns in Distributed Graph Frameworks

Large Scale Evolving Graphs with Burst Detection.

Continuous Queries for Multi-Relational Graphs

Bridging Distribution Gaps: Invariant Pattern Discovery for Dynamic Graph Learning

Ingress: an automated incremental graph processing system

On Reporting Durable Patterns in Temporal Proximity Graphs

Decoupled Graph Neural Networks for Large Dynamic Graphs

Exploring the Landscape of Distributed Graph Sketching

xDGP: A Dynamic Graph Processing System with Adaptive Partitioning

Adaptive Visual Scene Understanding: Incremental Scene Graph Generation

An In-Depth Study of Continuous Subgraph Matching (Complete Version)

High-Performance Massive Subgraph Counting Using Pipelined Adaptive-Group Communication.

GCG: Mining Maximal Complete Graph Patterns from Large Spatial Data

IncGraph:Large-Scale Graph Processing System for Real-Time and Incremental Computing