Abstract:Two important emerging trends are influencing the design, implementation and deployment of high performance parallel systems. The first is on the architectural end, where both economic and technological factors are compelling the use of off-the-shelf computing elements (workstations/PCs and networks) to put together high performance systems called clusters. The second is from the user community that is finding an increasing number of applications to benefit from such high performance systems. Apart from the scientific applications that have traditionally needed supercomputing power, a large number of graphics, visualization, database, web service and e-commerce applications have started using clusters because of their high processing and storage requirements. These applications have diverse characteristics and can place different Quality-of-Service (QoS) requirements on the underlying system (low response time, high throughput, high I/O demands, guaranteed response/throughput etc.). Further, clusters running such applications need to cater to potentially a large number of users (or other applications) in a time-shared manner. The underlying system needs to accommodate the requirements of each application, while ensuring that they do not interfere with each other.This paper focuses on the CPU resources of a cluster and investigates scheduling mechanisms to meet the responsiveness, throughput and guaranteed service requirements of different applications. Specifically, we propose and evaluate three different scheduling mechanisms. These mechanisms have been drawn from traditional solutions on parallel systems (gang scheduling and dynamic coscheduling), and have been extended to accommodate the new criteria under consideration. These mechanisms have been investigated using detailed simulation and workload models to show their pros and cons for different performance metrics.

Clustering based Probabilistic I/O Scheduling for Burst-Buffers Equipped HPC.

Time-sharing Parallel Applications Through Performance-Targeted Feedback-Controlled Real-Time Scheduling.

An Efficient Fault-Tolerant Scheduling Algorithm for Periodic Real-Time Tasks in Heterogeneous Platforms

Interference-Aware I/O Scheduling for Data-Intensive Applications on Hierarchical HPC Storage Systems

I/O Burst Prediction for HPC Clusters using Darshan Logs

I/O-Conscious And Prediction-Enabled Virtual Machines Scheduling

Fine-grained Policy-driven I/O Sharing for Burst Buffers

PPS: Fair and efficient black-box scheduling for multi-tenant GPU clusters

Scheduling Best-Effort and Real-Time Pipelined Applications on Time-Shared Clusters.

Design of PPU framework for processing ordered data blocks in the cluster environment

Task Scheduling Using Bayesian Optimization Algorithm for Heterogeneous Computing Environments

Intelligent colocation of HPC workloads

Task Scheduling for Heterogeneous Computing Based on Bayesian Optimization Algorithm

vPFS+: Managing I/O Performance for Diverse HPC Applications

Applications Classification and Scheduling on Heterogeneous HPC Systems Using Experimental Research.

BoPF: Mitigating the Burstiness-Fairness Tradeoff in Multi-Resource Clusters

Task Scheduling for GPU Heterogeneous Cluster.

A cluster-based scheduling model using SPT and SA for dynamic hybrid flow shop problems

Optimisation of job scheduling for supercomputers with burst buffers

Pvbts: A Novel Task Scheduling Algorithm For Heterogeneous Computing Platforms

Client-side Straggler-Aware I/O Scheduler for Object-based Parallel File Systems