Abstract:Big data processing is currently becoming increasingly important in modern era due to the continuous growth of the amount of data generated by various fields such as particle physics, human genomics, earth observation, etc. However, the efficiency of processing large-scale data on modern virtual infrastructure, especially on the virtualized cloud computing infrastructure, is not clear. This paper focuses on the performance of hadoop virtual cluster and proposes a scalable hadoop virtual cluster platform vHadoop for the large-scale MapReduce-based parallel data processing. We first describe the design and implementation of vHadoop platform. Then we perform a series of experiments to investigate both the static and dynamic performance of vHadoop platform, such as the performance characterization of cross-domain hadoop virtual cluster and live migraiton of hadoop virtual cluster. After that, we use the vHadoop platform to process 6 typical parallel clustering algorithms, such as Canopy, Dirichlet, Fuzzy k-Means, k-Means, Mean Shift, MinHash, etc, on two typical datasets. Experimental results verify the efficiency of vHadoop platform to process the MapReduce-based parallel machine learning applications.

HIVE: a Highly Scalable Framework for DVE

A scalable HLA-based distributed simulation framework for VR application

Developing a Scalable Distributed Simulation on HIVE

Efficient communication approach in HLA-based distributed simulation for VR application

Building High Performance DVR Via HLA, Scene Graph and Parallel Rendering.

A Scalable Semantic Grid Framework -- VDHA_Grid.

A Framework for Java 3d Based Collaborative Virtual Environment

MUDVE: Implement of A Multi-User Distributed Virtual Environment

An Effective Aggregation Method in Distributed Virtual Environments

A Latency-Adaptive Communication Architecture for Inter-Networked Virtual Environments

Implementation of Progressive Texture Transmission in Distributed Virtual Environments

On View Consistency in Multi-Server Distributed Virtual Environments

Research on Scalable Architecture of Remote Rendering

Framework Of Simulation Resources Management And Development In Distributed Virtual Environment

Vhadoop: A Scalable Hadoop Virtual Cluster Platform for MapReduce-Based Parallel Machine Learning with Performance Consideration

Scalable Distributed Standard Definition Video Conferencing System: Architecture and Forwarding Model

A Streaming-Based Approach for Remote Interaction of the Multi-Channel Display System for Group Users

Implementation Of Progressive Data Transmission In Distributed Virtual Environment

A Hierarchical Protocol Structure in Distributed Virtual Environment

On Designing and Testing Distributed Virtual Environments

A Scalable Framework for Large-Scale Distributed Collaboration