Abstract:The heterogeneous edge-cloud computing paradigm can provide a more optimal direction to deploy scientific workflows than traditional distributed computing or cloud computing environments. Due to the different sizes of scientific datasets and some of these datasets must keep private, it is still a difficult problem to finding an data placement strategy that can minimize data transmission as well as placement cost. To address this issue, this paper combines advantages of both edge and cloud computing to construct a data placement model, which can balance data transfer time and data placement cost using intelligent computation. The most difficult research challenge the model solved is to consider many constrain in this hybrid computing environments, which including shared datasets within individual and among multiple workflows across various geographical regions. According to the constructed model, the study propose a new data placement strategy named DE-DPSO-DPS, which using a discrete particle swarm optimization algorithm with differential evolution (DE-DPSO-DPA) to distribute these scientific datasets. The strategy also not only consider the characteristics such as the number and storage capacity of edge micro-datacenters, the bandwidth between different datacenters and the proportion of private datasets, but also analysis the performance of algorithm during the workflows execution. Comprehensive experiments are designed in simulated heterogeneous edge-cloud computing environments demonstrate that the data placement strategy can effectively reduce the data transmission time and placement cost as compared to traditional strategies for data-sharing scientific workflows.

Data Analysis and Synchronization on Inter-Continent Data Placement Laboratory

Distributed storage and processing of real-time data in smart grid dispatching and control systems

Data Placement and Task Scheduling Optimization for Data Intensive Scientific Workflow in Multiple Data Centers Environment.

Improving Performance for Geo-Distributed Data Process in Wide-Area.

A Strategy of Data Synchronization in Distributed System with Read Separating from Write

A Two-Step Data Placement and Task Scheduling Strategy for Optimizing Scientific Workflow Performance on Cloud Computing Platform

Analysis and Optimization of Massive Data Processing on High Performance Computing Architecture

Optimal Data Placement for Data-Sharing Scientific Workflows in Heterogeneous Edge-Cloud Computing Environments

Optimizing Data Placement in Heterogeneous Hadoop Clusters

A Fully Distributed Collection Technology for Mass Simulation Data

A Novel Data Placement Strategy for Data-Sharing Scientific Workflows in Heterogeneous Edge-Cloud Computing Environments

Scalable and Adaptive Data Replica Placement for Geo-Distributed Cloud Storages

Cost-Efficient Task Scheduling for Geo-distributed Data Analytics.

Distributed Heterogeneous Storage Based On Data Value

An Improved Data Placement Strategy in a Heterogeneous HadoopCluster

Efficient Data and Task Co-Scheduling for Scientific Workflow in Geo-Distributed Datacenters

A MapReduce Cluster Deployment Optimization Framework with Geo-distributed Data.

Adaptive priority-based data placement and multi-task scheduling in geo-distributed cloud systems

Towards Optimized Scheduling for Data‐intensive Scientific Workflow in Multiple Datacenter Environment

Block Storage Optimization and Parallel Data Processing and Analysis of Product Big Data Based on the Hadoop Platform

Infrastructures and services for remote sensing data production management across multiple satellite data centers