Abstract:Graph partitioning is a mandatory step in large-scale distributed graph processing. When partitioning real-world power-law graphs, the edge partitioning algorithm performs better than the traditional vertex partitioning algorithm, because it can cut a single vertex into multiple replicas to apportion the computation. Many advanced edge partitioning methods are designed for partitioning a static graph from scratch. However, the real-world graph structure changes continuously, which leads to a decrease in partition quality and affects the performance of the graph applications. Some studies are devoted to offline repartitioning or batch incremental partitioning, but how to deal with dynamics in real-time is still worthy of in-depth study. In this article, we discuss the impact of dynamic change on partition and discover that both insertion and deletion will lead to local suboptimal partitioning, which is the reason for the degradation of partition quality. As a solution, a dynamic edge partitioning algorithm is proposed to partition dynamics in real-time. Specifically, we deal with dynamics by a distributed stream and improve partition quality by reassigning some closely connected edges. Experiments show that it is robust to initial partition quality, dynamic scale and type, and distributed scale. Compared with the state-of-the-art dynamic partitioner, it can reduce vertex-cuts by 29.5 percent. Compared with the repartitioning algorithms, it can save the partitioning time by 91.0 percent. Applied on the graph task, it can reduce the increase of communication cost and the increase of the total time of task by 41.5 and 71.4 percent.

Partitioning Large-Scale Artificial Society on Distributed Cluster with Statistical Movement Graph

Grid-based partitioning for large-scale distributed agent-based crowd simulation

A Two-Tier Partition Algorithm for the Optimization of the Large-Scale Simulation of Information Diffusion in Social Networks.

CommPar: A Community-Based Model Partitioning Approach for Large-Scale Networked Social Dynamics Simulation.

A Task Partition Algorithm Based on Grid and Graph Partition for Distributed Crowd Simulation

An Environ ment Structu re Understanding Task Partition Method for Distributed Crowd Si mulation

Efficient Graph-Based Dynamic Load-Balancing for Parallel Large-Scale Agent-Based Traffic Simulation

An Optimized Partitioning Algorithm for Complex Network Based on Social Simulations on Cluster Computing Platform

A Framework of Evaluating Partitioning Mechanisms for Agent-Based Simulation Systems.

A Graph Partitioning Algorithm For Parallel Agent-Based Road Traffic Simulation

An Efficient Method for Graph Repartitioning in Distributed Environments.

Quasi-Streaming Graph Partitioning: A Game Theoretical Approach

A load balance algorithm for distributed crowd simulation based on connected graph partition

Graph partitioning method for social networks based on communication load balancing

Multilevel Task Partition Algorithm For Parallel Simulation Of Power System Dynamics

Optimization for Accelerating Large Scale Agent Based Simulation.

A Feasible Graph Partition Framework for Random Walks Implemented by Parallel Computing in Big Graph

Weighted Social Networks for A Large Scale Artificial Society

Group Reassignment for Dynamic Edge Partitioning

A Streaming Graph Partitioning Approach on Imbalance Cluster

A Distributed Graph Partitioning Algorithm for Processing Large Graphs