Abstract:N-hop neighborhoods information is very useful in analytic tasks on large-scale graphs, like finding clique in a social network, recommending friends or advertising links according to one’s interests, predicting links among websites and etc. To get the N-hop neighborhoods information on a large graph, such as a web graph, a twitter social graph, the most straightforward method is to conduct a breadth first search (BFS) on a parallel distributed graph processing framework, such as Pregel and GraphLab. However, due to the massive volume of message transfer, the BFS method results in high communication cost and has low efficiency. In this work, we propose a key/value based method, namely KVB, which perfectly fits into the prevailing parallel graph processing framework and computes N-hop neighborhoods on a large scale graph efficiently. Unlike the BFS method, our method need not transfer large amount of neighborhoods information, thus, significantly reduces the overhead on both the communication and intermediate results in the distributed framework.We formalize the N-hop neighborhoods query processing as an optimization problem based on a set of quantitative cost metrics of parallel graph processing. Moreover, we propose a solution to efficiently load only the relevant neighborhoods for computation. Specially, we prove the optimal partial neighborhoods load problem is NP-hard and carefully design a heuristic strategy. We have implemented our algorithm on a distributed graph framework- Spark GraphX and validated our solution with extensive experiments over a number of real world and synthetic large graphs on a modest indoor cluster. Experiments show that our solution generally gains an order of magnitude speedup comparing to the state-of-art BFS implementation.

Towards Efficient Path Query On Social Network With Hybrid Rdf Management

SparkRDF: In-Memory Distributed RDF Management Framework for Large-Scale Social Data.

Effective And Efficient Keyword Query Interpretation Using A Hybrid Graph

Distributed processing of regular path queries in RDF graphs

SparkRDF: Elastic Discreted RDF Graph Processing Engine with Distributed Memory

Partitioning Large-Scale Property Graph for Efficient Distributed Query Processing.

Scalable SPARQL Querying Using Path Partitioning

Scaling Hop-Based Reachability Indexing for Fast Graph Pattern Query Processing

GQARDF : A Graph-Based Approach Towards Efficient SPARQL Query Answering

An Efficient Parallel Algorithm of N-hop Neighborhoods on Graphs in Distributed Environment

Query Workload-based RDF Graph Fragmentation and Allocation

Gstore: Answering Sparql Queries Via Subgraph Matching

Adaptive Distributed RDF Graph Fragmentation and Allocation Based on Query Workload

RDF partitioning for scalable SPARQL query processing

Towards Efficient SPARQL Query Processing on RDF Data

Real Time Personalized Search on Social Networks

An Improved Distributed Query for Large-Scale RDF Data

High Throughput Shortest Distance Query Processing on Large Dynamic Road Networks

QRDF: an Efficient RDF Graph Processing System for Fast Query

Towards Efficient Distributed SPARQL Queries on Linked Data.

A Hybrid Approach Combining R*-Tree and k-d Trees to Improve Linked Open Data Query Performance