Abstract:In the past twenty years, we have witnessed an unprecedented production of data worldwide that has generated a growing demand for computing resources and has stimulated the design of computing paradigms and software tools to efficiently and quickly obtain insights on such a Big Data. State-of-the-art parallel computing techniques such as the MapReduce guarantee high performance in scenarios where involved computing nodes are equally sized and clustered via broadband network links, and the data are co-located with the cluster of nodes. Unfortunately, the mentioned techniques have proven ineffective in geographically distributed scenarios, i.e., computing contexts where nodes and data are geographically distributed across multiple distant data centers. In the literature, researchers have proposed variants of the MapReduce paradigm that obtain awareness of the constraints imposed in those scenarios (such as the imbalance of nodes computing power and of interconnecting links) to enforce smart task scheduling strategies. We have designed a hierarchical computing framework in which a context-aware scheduler orchestrates computing tasks that leverage the potential of the vanilla Hadoop framework within each data center taking part in the computation. In this work, after presenting the features of the developed framework, we advocate the opportunity of fragmenting the data in a smart way so that the scheduler produces a fairer distribution of the workload among the computing tasks. To prove the concept, we implemented a software prototype of the framework and ran several experiments on a small-scale testbed. Test results are discussed in the last part of the paper.

A new framework of cluster-based parallel processing system for high-performance geo-computing

CLUSTER-BASED OCEAN REMOTE SENSING IMAGE FUSION PARALLEL COMPUTING STRATEGY

An Effective Framework for Distributed Geospatial Query Processing in Grids

Parallel Spatial-Data Conversion Engine: Enabling Fast Sharing Of Massive Geospatial Data

GIS parallelizing system based on cluster system

The Framework of a Distributed File System for Geospatial Data Management.

Multi-screen Tiled Displayed, Parallel Rendering System for a Large Terrain Dataset

A Hierarchical Hadoop Framework to Process Geo-Distributed Big Data

A Model of Caching Geo-Data Sharing Based on Computer Cluster Technology

SDPPF — A MapReduce based parallel processing framework for spatial data

A Programming Framework Based on Multi-GPU

Research on Parallel Geo-Spatial Index Replication Strategy Based on PC Cluster System and its 3D Visualization Application

The Design and Implementation of Geographic Information Storage System Based on the Cloud Platform.

A generic framework for the development of geospatial processing pipelines on clusters

A new data-intensive parallel processing framework for spatial data

A Mechanism of Remote Sensing Image for Parallel Processing Base on Splitting Blocks

Study of High Performance Processing Platform Architecture in Remote Sensing

A New Approach to Improve the Cluster-Based Parallel Processing Efficiency of High-Resolution Remotely Sensed Image

A Novel Approach For Geospatial Computational Task Processing In Grid Environment

An Efficient and Scalable Framework for Processing Remotely Sensed Big Data in Cloud Computing Environments

A MapReduce Computing Framework Based on GPU Cluster.