Abstract:• A task-parallel framework for local structural vertex similarity calculation. • A threshold-adaptive technique to execute a similarity calculation task. • A Spark-based implementation with three optimization techniques. • Comprehensive performance evaluation of the vertex similarity calculation strategies. In many graph analytical applications, the local structural vertex similarity calculation is an essential prerequisite for advanced graph mining. The similarity calculation finds out all the similar vertex pairs whose local structural similarity scores (like the number of common neighbors, and the Jaccard index of adjacency sets) are above a given threshold. The real-world applications use a wide range of similarity thresholds. However, the existing distributed methods for the problem only optimize for either high thresholds (> 0.7) or low thresholds (< 0.1). To overcome the drawback, we propose a new distributed vertex similarity calculation framework VSIM that is efficient under a broad range of thresholds. VSIM processes static undirected graphs with local structural similarity scores that measure the similarity between vertices based on the first-order topology information. VSIM generates a similarity calculation task for every vertex in the graph and conducts all the tasks in parallel on a distributed computing platform along with a distributed key-value store. Each task finds vertices similar to a given center vertex with two task execution modes. The two modes optimize for high and low thresholds, respectively. Each task picks the suitable mode adaptively according to cost estimation models. We also propose an efficient implementation for VSIM on Apache Spark, with three optimization techniques to reduce communication costs and balance workloads on power-law graphs. The experimental evaluation shows that VSIM outperforms the state-of-the-art distributed methods by up to 67x speedup. VSIM can achieve near-linear node scalability in low-threshold and small cache scenarios.

Vertex dissimilarity in community discovery

Weighted-Ring Similarity Measurement for Community Detection in Social Network

GMAC: A Seed-Insensitive Approach to Local Community Detection

Toward Seed-Insensitive Solutions to Local Community Detection.

A New Vertex Similarity Metric for Community Discovery: a Local Flow Model

Study on Similarity Based on Connection Degree in Social Network

A vertex similarity probability model for finding network community structure

Measuring Similarity Between Vertices and Its Application in Social Network

A Vertex Similarity Index Using Community Information to Improve Link Prediction Accuracy

Local Community Detection Using Link Similarity

Group Similarity Based Algorithm for Network Community Structure Detection

Finding Community Structure Based on Subgraph Similarity

Scalable Community Discovery of Large Networks

Community Detection by Neighborhood Similarity

Community Detection in Social Networks Employing Component Independency

VSIM: Distributed Local Structural Vertex Similarity Calculation on Big Graphs

Dissimilarity-based hypothesis testing for community detection in heterogeneous networks

Latent Geometry Inspired Graph Dissimilarities Enhance Affinity Propagation Community Detection in Complex Networks

Community Detection by Affinity Propagation

A Community Detection Algorithm Based on Local Similarity

A community detection algorithm based on multi-similarity method