Abstract:Data-intensive applications are becoming increasingly common in Grid environments. These applications require enormous volume of data for the computation. Most conventional meta-scheduling approaches are aimed at computation intensive application and they do not take data requirement of the applications into account, thus leading to poor performance. Efficient scheduling of data-intensive applications in Grid environments is a challenging problem. In addition to process utilization and average turnaround time, it is important to consider the worst-case turnaround time in evaluating the performance of Grid scheduling strategies. In this paper, we propose an adaptive scheduling scheme that takes into account both the computational requirements and the data requirements of the jobs while making scheduling decisions. In our scheme, data transfer is viewed in par with computation and explicitly considered when scheduling. Jobs are dispatched to the sites that are optimal in terms of both data transfer time and computation time. In addition, our scheme overlaps a job's data transfer time with its own queuing time and other jobs' computation time as much as possible. Trace-based simulations show that the proposed scheme can gain significant performance benefits for data-intensive jobs.

Agile Data Streaming for Grid Applications

Scheduling On-demand Data Streaming for Cyberinfrastructure Applications with Constraints of Storage and Bandwidth

Grid Service-Based Parallel Finite Element Analysis

An Integrated Resource Management and Scheduling System for Grid Data Streaming Applications

Block-Based Concurrent and Storage-Aware Data Streaming for Grid Applications with Lots of Small Files

Real Time Streaming Data Grid Applications

Storage Aware Resource Allocation for Grid Data Streaming Pipelines

Concurrent and Storage-Aware Data Streaming for Data Processing Workflows in Grid Environments

A web-based system for adaptive data transfer in grid

An Adaptive System of Service-Oriented Visualization on the Grid

A Static Resource Allocation Framework for Grid-based Streaming Applications.

GSAF: A Grid-based Services Transfer Framework

Intelligent File Transfer Protocol for Grid Environment

An Uniform Access Method of Heterogeneous Big Data with Power Grid Application

A Data-Driven Architecture Design of Stream Computing for the Dispatch and Control System of the Power Grid

Data Transfer Based on Multiple Replicas in the Grid Environment

Resource Allocation in a Middleware for Streaming Data

Gates: A Grid-Based Middleware for Processing Distributed Data Streams

An adaptive strategy for scheduling data-intensive applications in Grid environments

A Data Streams Analysis Strategy Based on Hadoop Scheduling Optimization for Smart Grid Application.

Management of Real-Time Streaming Data Grid Services