Special Issue on Scheduling in Multiprogrammed Parallel Systems

K. Sevcik

DOI: https://doi.org/10.1145/309746.581197

1999-03-01

Abstract:Large-scale computation facilities have become primarily dependent on multiprocessors to support their workloads. This is true at the major "Supercomputer" centers, such as NCSA and SDSC, as well as at a very large number of other shared computational facilities. These environments have the common characteristic that they support a diverse population of users who submit jobs with resource requirements that span an immense range. Job scheduling in parallel systems became a research focus of numerous groups star t ing about ten years ago. While much good and interesting research was done on the topic, it had little influence on practice in computing environments, since some of the assumptions commonly made were inconsistent with existing practices. While the research studies investigated sophisticated scheduling rules, ofter assuming a great deal of flexibility and extensive knowledge about individual jobs, scheduling in practice was done by fundamental schedulers that activated jobs in fixed sized partit ions on a FCFS or possibly priority basis. In the past five years or so, there has been a trend of convergence between the scheduling approaches studied in research and those applied in practice. There has been a great deal of research into appropriate workload models and good job scheduling algorithms for practical environments, based on the realization that the best use of shared resources does not typically result when resource allocation decisions are based exclusively on the users' request. Many excellent papers relating job scheduling theory to practice have appeared in a series of workshops organized by Dror Feitelson and Larry Rudolph. (The proceedings of these workshops are cited as references 7 to 10 in the paper by Dowdy et al.) This special issue of Performance Evaluation Review contains five papers relating to various aspects of the scheduling problem in multiprogrammed parallel systems. All the authors have made notable previous contributions to this topic, and here, they have been invited to present some of their more recent thoughts on the topic. The paper by Downey and Feitelson, "The Elusive Goal of Workload Characterization", is based on measurement da ta obtained from four different supercomputing sites. This data is used to investigate how workloads on parallel computing systems should be characterized. They seek good models for characterizing service time distributions, and observe correlations that commonly exist between pairs of job characteristics. Setia, Squillante, and Naik, in their paper, "Impact of Job Memory Requirements on Gang-Scheduling Performance", use measurements of job memory requirements obtained at the San Diego Supercomputer Center. They create models to investigate the performance of gang-scheduling with both processor and memory requirements taken into account. The paper by Chan, Dandamudi, and Majumdar , "Experiences with Parallel Job Scheduling on a Transputer System", describes an empirical s tudy of the relative performance of time-sharing and space-sharing scheduling approaches. They consider different interconnection network topologies and different software architectures. Squillante, Yao and Zhang's paper, "The Impact of Job Arrival Pat terns on Parallel Job Scheduling", uses job arrival pat terns observed at the Cornell Theory Center to establish an improved model of the arrival process. The new model is employed to investigate how modelled response times are affected under various job scheduling strategies. Dowdy, Rosti, Serazzi snd Smirni, in their paper, "Scheduling Issues in High-Performance Computing" examine some of the pedagogical issues that arise in job scheduling in parallel processing systems ranging from massive multiprocessors to networks of workstations. They present and counter some criticisms that have been made about the utility of modelling approaches in studying scheduling problems. I sincerely thank all the authors for their hard work on the papers that they have contributed to this special issue. I hope all you readers will find the papers both interesting and useful to your future work.