Abstract:Out-of-core random walk system has recently attracted a lot of attention as an economical way to run billions of walkers over large graphs. However, existing out-of-core random walk systems are all built upon general out-of-core graph processing frameworks, and hence do not take advantage of the unique properties of random walk applications. Different from traditional graph analysis algorithms, the sampling process of random walk can be decoupled from the processing of the walkers. It enables the system to reserve only pre-sample results in memory, which are typically much smaller than the entire edge set. Moreover, in random walk, it is not the number of walkers but the number of steps moved per second that dominates the overall performance. Thus, with independent walkers, there is no need to process all the walkers simultaneously. In this paper, we present NosWalker, an out-of-core random walk system that replaces the graph oriented scheduling with a decoupled system architecture that provides walker oriented scheduling. NosWalker is able to adaptively generate walkers and flexibly adjust the distribution of reserved pre-sample results in memory. Instead of processing all the walkers at once, NosWalker only tries its best to keep a few walkers able to continuously move forward. Experimental results show that NosWalker can achieve up to two orders of magnitude speedup compared to state-of-the-art out-of-core random walk systems. In particular, NosWalker demonstrates superior performance when the memory capacity can only hold about 10%-50% of the graph data, which can be a common case when the user needs to run billions of walkers over large graphs.

KnightKing

GraphWalker: an I/O-Efficient and Resource-Friendly Graph Analytic System for Fast and Scalable Random Walks.

Random Walk and Feedback on Scholarly Network.

Toward Fast and Scalable Random Walks over Disk-Resident Graphs Via Efficient I/O Management.

Random Walks on Huge Graphs at Cache Efficiency

Revisiting Random Walks for Learning on Graphs

Random Walks: A Review of Algorithms and Applications

Random Walk on Multiple Networks

Repelling Random Walks

A general framework for estimating graphlet statistics via random walk

Walking with Perception: Efficient Random Walk Sampling via Common Neighbor Awareness

NosWalker: A Decoupled Architecture for Out-of-Core Random Walk Processing.

Random vicious walks and random matrices

UniWalk: Unidirectional Random Walk Based Scalable SimRank Computation over Large Graph

High-Throughput GPU Random Walk with Fine-Tuned Concurrent Query Processing

SmartWalk: Enhancing Social Network Security via Adaptive Random Walks

A Survey on Recent Random Walk-based Methods for Embedding Knowledge Graphs

FlowWalker: A Memory-efficient and High-performance GPU-based Dynamic Graph Random Walk Framework

Random-Walk Graph Embeddings and the Influence of Edge Weighting Strategies in Community Detection Tasks

Frustrated Random Walks: A Fast Method to Compute Node Distances on Hypergraphs

Detecting Community Structure for Undirected Big Graphs Based on Random Walks