Abstract:• A task-parallel framework for local structural vertex similarity calculation. • A threshold-adaptive technique to execute a similarity calculation task. • A Spark-based implementation with three optimization techniques. • Comprehensive performance evaluation of the vertex similarity calculation strategies. In many graph analytical applications, the local structural vertex similarity calculation is an essential prerequisite for advanced graph mining. The similarity calculation finds out all the similar vertex pairs whose local structural similarity scores (like the number of common neighbors, and the Jaccard index of adjacency sets) are above a given threshold. The real-world applications use a wide range of similarity thresholds. However, the existing distributed methods for the problem only optimize for either high thresholds (> 0.7) or low thresholds (< 0.1). To overcome the drawback, we propose a new distributed vertex similarity calculation framework VSIM that is efficient under a broad range of thresholds. VSIM processes static undirected graphs with local structural similarity scores that measure the similarity between vertices based on the first-order topology information. VSIM generates a similarity calculation task for every vertex in the graph and conducts all the tasks in parallel on a distributed computing platform along with a distributed key-value store. Each task finds vertices similar to a given center vertex with two task execution modes. The two modes optimize for high and low thresholds, respectively. Each task picks the suitable mode adaptively according to cost estimation models. We also propose an efficient implementation for VSIM on Apache Spark, with three optimization techniques to reduce communication costs and balance workloads on power-law graphs. The experimental evaluation shows that VSIM outperforms the state-of-the-art distributed methods by up to 67x speedup. VSIM can achieve near-linear node scalability in low-threshold and small cache scenarios.

A Vertex-Centric Graph Simulation Algorithm for Large Graphs.

An Efficient Parallel Algorithm of N-hop Neighborhoods on Graphs in Distributed Environment

Vertex-Centric Graph Processing: The Good, the Bad, and the Ugly

Vertex-centric Parallel Algorithms for Identifying Key Vertices in Large-Scale Graphs

Tuning the granularity of parallelism for distributed graph processing

Distributed Graph Pattern Matching

Graphu: A Unified Vertex-Centric Parallel Graph Processing Platform

Parallel Algorithms for Flexible Pattern Matching on Big Graphs

An Efficient Graph Processing System

DFA-G: A Unified Programming Model for Vertex-Centric Parallel Graph Processing

Distributed Graph Simulation: Impossibility and Possibility.

ScaleG: A Distributed Disk-based System for Vertex-centric Graph Processing

VSIM: Distributed Local Structural Vertex Similarity Calculation on Big Graphs

Thinking Like a Vertex: a Survey of Vertex-Centric Frameworks for Distributed Graph Processing

Optimizing Graph Processing on GPUs

SPFC: an Effective Optimization for Vertex-Centric Graph Processing Systems

Graph Computing Systems for Large Scale Graph Analysis

A Novel Clustering Algorithm on Large-Scale Graph Data

A Survey of Distributed Graph Algorithms on Massive Graphs