Abstract:A fundamental building block in any graph algorithm is a graph container - - a data structure used to represent the graph. Ideally, a graph container enables efficient access to the underlying graph, has low space usage, and supports updating the graph efficiently. In this paper, we conduct an extensive empirical evaluation of graph containers designed to support running algorithms on large graphs. To our knowledge, this is the first apples-to-apples comparison of graph containers rather than overall systems, which include confounding factors such as differences in algorithm implementations and infrastructure. We measure the running time of 10 highly-optimized algorithms across over 20 different containers and 10 graphs. Somewhat surprisingly, we find that the average algorithm running time does not differ much across containers, especially those that support dynamic updates. Specifically, a simple container based on an off-the-shelf B-tree is only 1.22× slower on average than a highly optimized static one. Moreover, we observe that simplifying a graph-container Application Programming Interface (API) to only a few simple functions incurs a mere 1.16× slowdown compared to a complete API. Finally, we also measure batch-insert throughput in dynamic-graph containers for a full picture of their performance. To perform the benchmarks, we introduce BYO, a unified framework that standardizes evaluations of graph-algorithm performance across different graph containers. BYO extends the Graph Based Benchmark Suite (Dhulipala et al. 18), a state-of-the-art graph algorithm benchmark, to easily plug into different dynamic graph containers and enable fair comparisons between them on a large suite of graph algorithms. While several graph algorithm benchmarks have been developed to date, to the best of our knowledge, BYO is the first system designed to benchmark graph containers.

BYO: A Unified Framework for Benchmarking Large-Scale Graph Containers

BYO: A Unified Framework for Benchmarking Large-Scale Graph Containers

A Dwarf-based Scalable Big Data Benchmarking Methodology

Evaluation and Analysis of Distributed Graph-Parallel Processing Frameworks

A Comparative Study of Containers and Virtual Machines in Big Data Environment

Benchmarking Graph Data Management and Processing Systems: A Survey

Evaluation and Trade-offs of Graph Processing for Cloud Services

GC-Bench: An Open and Unified Benchmark for Graph Condensation

GARDENIA: A Domain-specific Benchmark Suite for Next-generation Accelerators

HUGE: an Efficient and Scalable Subgraph Enumeration System

A caving degree approach for the single container loading problem

Benchmarking Big Data Systems: State-of-the-Art and Future Directions

The ParClusterers Benchmark Suite (PCBS): A Fine-Grained Analysis of Scalable Graph Clustering

Benchmarking Graph Conformal Prediction: Empirical Analysis, Scalability, and Theoretical Insights

Comparison Between Chunk-Based and Layer-Based Container Image Storage Approaches: an Empirical Study

A Feasible Graph Partition Framework for Parallel Computing of Big Graph

A Comprehensive Graph Pooling Benchmark: Effectiveness, Robustness and Generalizability

Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights

XFlow: Benchmarking Flow Behaviors over Graphs

Benchmark Frameworks and Τbench

Stream Bench: Towards Benchmarking Modern Distributed Stream Computing Frameworks