Abstract:With the huge demand for graph analytics in many real-world applications, massive iterative graph processing jobs are concurrently performed on the same graphs and suffer from significant high data access cost. To lower the data access cost toward high performance, several out-of-core concurrent graph processing solutions are recently designed to handle concurrent jobs by enabling these jobs to share the accesses of the same graph data. However, the set of active vertices in each partition are usually different for various concurrent jobs and also evolve with time, where some high-degree ones (or called hub-vertices) of these active vertices require more iterations to converge due to the power-law property of real-world graphs. In consequence, existing solutions still suffer from much unnecessary I/O traffic, because they have to entirely load each partition into the memory for concurrent jobs even if most vertices in this partition are inactive and may be shared by a few jobs. In this paper, we propose an efficient structure-aware storage system, called GraphSO, for higher throughput of the execution of concurrent graph processing jobs. It can be integrated into existing out-of-core graph processing systems to promote the execution efficiency of concurrent jobs with lower I/O overhead. The key design of GraphSO is a fine-grained storage management scheme. Specifically, it logically divides the partitions of existing graph processing systems into a series of small same-sized chunks. At runtime, these small chunks with active vertices are judiciously loaded by GraphSO to construct new logical partitions (i.e., each logical partition is a subset of active chunks) for existing graph processing systems to handle, where the most-frequently-used chunks are preferentially loaded to construct the logical partitions and the other ones are delayed to wait to be required by more jobs. In this way, it can effectively spare the cost of loading the graph data as-ociated with the inactive vertices with low repartitioning overhead and can also enable the loaded graph data to be fully shared by concurrent jobs. Moreover, GraphSO also designs a buffering strategy to efficiently cache the most-frequently-used chunks in the main memory to further minimize the I/O traffic by avoiding repeated load of them. Experimental results show that GraphSO improves the throughput of GridGraph, GraphChi, X-Stream, DynamicShards, LUMOS, Graphene, and Wonderland by 1.4-3.5 times, 2.1-4.3 times, 1.9-4.1 times, 1.9-2.9 times, 1.5-3.1 times, 1.3-1.5 times, and 1.3-2.7 times after integrating with them, respectively.

Memory System Optimization for Graph Processing: a Survey

Graph Algorithm Optimization for Spintronics-based In-memory Computing Architecture

A Structure-aware Storage Optimization for Out-of-Core Concurrent Graph Processing

An Efficient Graph Accelerator with Distributed On-Chip Memory Hierarchy.

Optimizing Graph Processing on GPUs

A Distributed Graph Data Storage and Computing Framework

Cacheap: Portable and Collaborative I/O Optimization for Graph Processing

A Survey of GPGPU Parallel Processing Architecture Performance Optimization

GraphH: A Processing-in-Memory Architecture for Large-Scale Graph Processing

A Comprehensive Reconfigurable Computing Approach to Memory Wall Problem of Large Graph Computation

Memory-Based High-Performance Optimization for High Concurrent Data-Intensive Problems

NGraph: Parallel Graph Processing in Hybrid Memory Systems

An Energy-Efficient In-Memory Accelerator for Graph Construction and Updating

An Improved Memory Management Scheme for Large Scale Graph Computing Engine GraphChi.

Accelerating Graph-Connected Component Computation with Emerging Processing-In-Memory Architecture.

SoGraph: A State-Aware Architecture for Out-of-Memory Graph Processing on HBM-Equipped FPGAs

Optimizing Memory Efficiency of Graph NeuralNetworks on Edge Computing Platforms

A Survey of Memory-Centric Energy Efficient Computer Architecture

Software Systems Implementation and Domain-Specific Architectures Towards Graph Analytics

A Survey of Graph Pre-processing Methods: From Algorithmic to Hardware Perspectives

Towards High-Performance Graph Processing: From a Hardware/Software Co-Design Perspective