Abstract:In the past twenty years, we have witnessed an unprecedented production of data worldwide that has generated a growing demand for computing resources and has stimulated the design of computing paradigms and software tools to efficiently and quickly obtain insights on such a Big Data. State-of-the-art parallel computing techniques such as the MapReduce guarantee high performance in scenarios where involved computing nodes are equally sized and clustered via broadband network links, and the data are co-located with the cluster of nodes. Unfortunately, the mentioned techniques have proven ineffective in geographically distributed scenarios, i.e., computing contexts where nodes and data are geographically distributed across multiple distant data centers. In the literature, researchers have proposed variants of the MapReduce paradigm that obtain awareness of the constraints imposed in those scenarios (such as the imbalance of nodes computing power and of interconnecting links) to enforce smart task scheduling strategies. We have designed a hierarchical computing framework in which a context-aware scheduler orchestrates computing tasks that leverage the potential of the vanilla Hadoop framework within each data center taking part in the computation. In this work, after presenting the features of the developed framework, we advocate the opportunity of fragmenting the data in a smart way so that the scheduler produces a fairer distribution of the workload among the computing tasks. To prove the concept, we implemented a software prototype of the framework and ran several experiments on a small-scale testbed. Test results are discussed in the last part of the paper.

A security framework in G-Hadoop for big data computing across distributed Cloud data centres

A General Framework to Design Secure Cloud Storage Protocol Using Homomorphic Encryption Scheme.

G-Hadoop: MapReduce across distributed data centers for data-intensive computing

Comparative Analysis of Hadoop Security Ad-Ons

Comprehensive Security Management System for Hadoop Platforms

A Distributed Data Mining System Framework for Mobile Internet Access Log Based on Hadoop.

Research on Data Security Policy of Hadoop Cloud Computing Platform for Large-scale Critical Business

A Hierarchical Hadoop Framework to Process Geo-Distributed Big Data

Implementing Privacy Preserving Techniques in Distributed Big Data Framework on Cloud Computing

Design of a Trusted File System Based on Hadoop

Design and Implementation of Data Encryption in Cloud Based on HDFS

Shiyf: A Secured And High-Integrity Yarn Framework

A New Data Access Mechanism for HDFS

Security and Privacy Aspects in MapReduce on Clouds: A Survey

A Novel Hadoop Security Model for Addressing Malicious Collusive Workers

HadoopM: A Message-Enabled Data Processing System on Large Clusters.

HPACS: A High Privacy and Availability Cloud Storage Platform with Matrix Encryption

Cloud Computing Security Analysis and Reinforcement of Hadoop Environment

The Design and Implementation of Geographic Information Storage System Based on the Cloud Platform.

Eavesdropper: A Framework for Detecting the Location of the Processed Result in Hadoop.

Study and implementation of block-encryption permutation algorithm based on Hadoop platform