Abstract:With the increasing availability of graph data and widely adopted cloud computing paradigm, graph partitioning has become an efficient pre-processing technique to balance the computing workload and cope with the large scale of input data. Since the cost of partitioning the entire graph is strictly prohibitive, there are some recent tentative works towards streaming graph partitioning which run faster, are easily parallelized, and can be incrementally updated. Most of the existing works on streaming partitioning assume that worker nodes within a cluster are homogeneous in nature. Unfortunately, this assumption does not always hold. Experiments show that these homogeneous algorithms suffer a significant performance degradation when running at heterogeneous environment. In this paper, we propose a novel adaptive streaming graph partitioning approach to cope with heterogeneous environment. We first formally model the heterogeneous computing environment with the consideration of the unbalance of computing ability (e.g., the CPU frequency) and communication ability (e.g., the network bandwidth) for each node. Based on this model, we propose a new graph partitioning objective function that aims to minimize the total execution time of the graph-processing job. We then explore some simple yet effective streaming algorithms for this objective function that can achieve balanced and efficient partitioning result. Extensive experiments are conducted on a moderate sized computing cluster with real-world web and social network graphs. The results demonstrate that the proposed approach achieves significant improvement compared with the state-of-the-art solutions.

An Environ ment Structu re Understanding Task Partition Method for Distributed Crowd Si mulation

A Task Partition Algorithm Based on Grid and Graph Partition for Distributed Crowd Simulation

PaCS: A Parallel Computation Framework for Field-Based Crowd Simulation

Partitioning Large-Scale Artificial Society on Distributed Cluster with Statistical Movement Graph

CommPar: A Community-Based Model Partitioning Approach for Large-Scale Networked Social Dynamics Simulation.

Grid-based partitioning for large-scale distributed agent-based crowd simulation

Field-based crowd simulation

Informative Scene Decomposition for Crowd Analysis, Comparison and Simulation Guidance

A load balance algorithm for distributed crowd simulation based on connected graph partition

Review on Parallel Crowd Simulation

Visually-Guided Pedestrian Crowd Simulation

Multilevel Task Partition Algorithm For Parallel Simulation Of Power System Dynamics

A simulation task partition method based on cloud computing resource prediction using ensemble learning

A Mechanism of Remote Sensing Image for Parallel Processing Base on Splitting Blocks

A hierarchical framework for crowd behavior simulation with constraints on different levels

Heterogeneous Environment Aware Streaming Graph Partitioning

Parallel Multilayer Particle Collision Detection Method Based on Performance Estimation

A Collaborative Simulation Framework based on Distributed Computing Environment

A Hybrid Model for Simulation of Crowd Evacuation

Dense Crowd Simulation Based on Continuum Model and Aggregate Dynamics Model

Distributed parallel modeling for traffic simulation