Abstract:Community discovery plays an essential role in the analysis of the structural features of complex networks. Since online networks grow increasingly large and complex over time, the methods traditionally used for community discovery cannot efficiently handle large-scale network data. This introduces the important problem of how to effectively and efficiently discover large communities from complex networks. In this study, we propose a fast parallel community discovery model called picaso (a parallel community discovery a lgorithm based on approximate optimization), which integrates two new techniques: (1) Mountain model, which works by utilizing graph theory to approximate the selection of nodes needed for merging, and (2) Landslide algorithm, which is used to update the modularity increment based on the approximated optimization. In addition, the GraphX distribution computing framework is employed in order to achieve parallel community detection over complex networks. In the proposed model, clustering on modularity is used to initialize the Mountain model as well as to compute the weight of each edge in the networks. The relationships among the communities are then simplified by applying the Landslide algorithm, which allows us to obtain the community structures of the complex networks. Extensive experiments were conducted on real and synthetic complex network datasets, and the results demonstrate that the proposed algorithm can outperform the state of the art methods, in effectiveness and efficiency, when working to solve the problem of community detection. Moreover, we demonstratively prove that overall time performance approximates to four times faster than similar approaches. Effectively our results suggest a new paradigm for large-scale community discovery of complex networks.

Improvement of path analysis algorithm in social networks based on HBase

Research on the Social Network Search Strategy from the Viewpoint of Comprehensive Influence Maximization

Towards Efficient Path Query On Social Network With Hybrid Rdf Management

Large-Scale Social Network Analysis Based on MapReduce

Evaluating Large Graph Processing in MapReduce Based on Message Passing

Scaling Hop-Based Reachability Indexing for Fast Graph Pattern Query Processing

Exploratory Analysis for Big Social Data Using Deep Network

A Two-Tier Partition Algorithm for the Optimization of the Large-Scale Simulation of Information Diffusion in Social Networks.

Detecting Overlapping Community Structures In Networks With Global Partition And Local Expansion

Hub-Accelerator: Fast and Exact Shortest Path Computation in Large Social Networks

HBaseSpatial: A Scalable Spatial Data Storage Based on HBase

An Efficient Shortest Path Approach for Social Networks Based on Community Structure

Community structure mining in big data social media networks with MapReduce

Research on big data mining based on improved parallel collaborative filtering algorithm

Power Big Data Analysis Platform Design Based on Hadoop

Understanding Parallelism in Graph Traversal on Multi-Core Clusters

An improved belief propagation algorithm for detecting meso-scale structure in complex networks

Optimizing Data Accesses for Breadth-First Search on Shared Memory Computers.

An improved density peaks-based clustering method for social circle discovery in social networks

A Progressive Approach for Neighboring Geosocial Communities Search over Large Spatial Graphs

A Fast Parallel Community Discovery Model on Complex Networks Through Approximate Optimization