Abstract:With the huge demand for graph analytics in many real-world applications, massive iterative graph processing jobs are concurrently performed on the same graphs and suffer from significant high data access cost. To lower the data access cost toward high performance, several out-of-core concurrent graph processing solutions are recently designed to handle concurrent jobs by enabling these jobs to share the accesses of the same graph data. However, the set of active vertices in each partition are usually different for various concurrent jobs and also evolve with time, where some high-degree ones (or called hub-vertices) of these active vertices require more iterations to converge due to the power-law property of real-world graphs. In consequence, existing solutions still suffer from much unnecessary I/O traffic, because they have to entirely load each partition into the memory for concurrent jobs even if most vertices in this partition are inactive and may be shared by a few jobs. In this paper, we propose an efficient structure-aware storage system, called GraphSO, for higher throughput of the execution of concurrent graph processing jobs. It can be integrated into existing out-of-core graph processing systems to promote the execution efficiency of concurrent jobs with lower I/O overhead. The key design of GraphSO is a fine-grained storage management scheme. Specifically, it logically divides the partitions of existing graph processing systems into a series of small same-sized chunks. At runtime, these small chunks with active vertices are judiciously loaded by GraphSO to construct new logical partitions (i.e., each logical partition is a subset of active chunks) for existing graph processing systems to handle, where the most-frequently-used chunks are preferentially loaded to construct the logical partitions and the other ones are delayed to wait to be required by more jobs. In this way, it can effectively spare the cost of loading the graph data as-ociated with the inactive vertices with low repartitioning overhead and can also enable the loaded graph data to be fully shared by concurrent jobs. Moreover, GraphSO also designs a buffering strategy to efficiently cache the most-frequently-used chunks in the main memory to further minimize the I/O traffic by avoiding repeated load of them. Experimental results show that GraphSO improves the throughput of GridGraph, GraphChi, X-Stream, DynamicShards, LUMOS, Graphene, and Wonderland by 1.4-3.5 times, 2.1-4.3 times, 1.9-4.1 times, 1.9-2.9 times, 1.5-3.1 times, 1.3-1.5 times, and 1.3-2.7 times after integrating with them, respectively.

SpeedCore: Space-efficient and Dependency-aware GPU Parallel Framework for Core Decomposition

Efficient Parallel D-Core Decomposition at Scale

PICO: Accelerating All k-Core Paradigms on GPU

I/O Efficient Core Graph Decomposition: Application to Degeneracy Ordering.

Connectivity-Based Segmentation for GPU-Accelerated Mesh Decompression

I/O Efficient Core Graph Decomposition at Web Scale

Parallel Batch-Dynamic Algorithms for $k$-Core Decomposition and Related Graph Problems

Mesh Segmentation for Parallel Decompression on GPU.

Clip: A Disk I/O Focused Parallel Out-of-Core Graph Processing System

Fast Multilayer Core Decomposition and Indexing

HyTGraph: GPU-Accelerated Graph Processing with Hybrid Transfer Management

Hybrid CPU-GPU Framework for Network Motifs

Core Decomposition on Uncertain Graphs Revisited

PRODA: Improving Parallel Programs on GPUs Through Dependency Analysis

Experimental Evaluation of Distributed k-Core Decomposition

A Structure-aware Storage Optimization for Out-of-Core Concurrent Graph Processing

Cross-Core Data Sharing for Energy-Efficient GPUs

Garaph: Efficient GPU-accelerated Graph Processing on a Single Machine with Balanced Replication.

Subway: minimizing data transfer during out-of-GPU-memory graph processing

A Parallel Strategy to Accelerate Neighborhood Operation for Raster Data Coordinating CPU and GPU

Efficient Graph Hierarchical Decomposition with User Engagement and Tie Strength