Abstract:With Dynamic Resource Management (DRM) the resources assigned to a job can be changed dynamically during its execution. From the system's perspective, DRM opens a new level of flexibility in resource allocation and job scheduling and therefore has the potential to improve system efficiency metrics such as the utilization rate, job throughput, energy efficiency, and responsiveness. From the application perspective, users can tailor the resources they request to their needs offering potential optimizations in queuing time or charged costs. Despite these obvious advantages and many attempts over the last decade to establish DRM in HPC, it remains a concept discussed in academia rather than being successfully deployed on production systems. This stems from the fact that support for DRM requires changes in all the layers of the HPC system software stack including applications, programming models, process managers, and resource management software, as well as an extensive and holistic co-design process to establish new techniques and policies for scheduling and resource optimization. In this work, we therefore start with the assumption that resources are accessible by processes executed either on them (e.g., on CPU) or controlling them (e.g., GPU-offloading). Then, the overall DRM problem can be decomposed into dynamic process management (DPM) and dynamic resource mapping or allocation (DRA). The former determines which processes (or which change in processes) must be managed and the latter identifies the resources where they will be executed. The interfaces for such \mbox{DPM/DPA} in these layers need to be standardized, which requires a careful design to be interoperable while providing high flexibility. Based on a survey of existing approaches we propose design principles, that form the basis of a holistic approach to DMR in HPC and provide a prototype implementation using MPI.

The Design and Implementation of Distributed Resource Manager in Optical Grid Networks

Scalable Resource Management And Load Assignment For Grid And Peer-To-Peer Services

An Architecture Of Game Grid Based On Resource Router

Distributed resource scheduling based on multi-node negotiation for grid systems

Architecture Design of Wide-area Distributed Real-time Database in Grid Dispatching Control System

Joint Scheduling and Routing for QoS Guaranteed Packet Transmission in Energy Efficient Reconfigurable WDM Mesh Networks

A Novel WSRF and Multi-Agent based Distributed System Resource Management Scheme

Research of the Grid Computing System Applied in Optical Simulation

Design And Implementation Of A Novel Grid-Vo-Enabled Compound Resource Tree Management Model In Multimedia Content Delivery Networks

Robust and cost-effective architecture design for smart grid communications: A multi-stage middleware deployment approach

A Novel Routing and Wavelength Assignment Algorithm for Multicast in Optical Grid Networks

An Integrated Resource Management and Scheduling System for Grid Data Streaming Applications

Distributed Resource Allocation and Load Management for Smart Grid-Powered Edge Computing Systems

GOD Queuing: A Novel Grid-based Resource Co-allocation Scheme in Distributed Multimedia System

Resource Virtualization and Universal Routing Architecture for Heterogeneous Optical Networks

Implementing Available Bandwidth Detection and Adaption: A Spectrum-Resource-Diffusion Routing Scheme in Flexible Grid Optical Networks

DQP:A Distributed Query Processor on Grid

Comprehensive Analysis of Performance, Fault-Tolerance and Scalability in Grid Resource Management System

Design Principles of Dynamic Resource Management for High-Performance Parallel Programming Models

A DHT-based grid monitoring system

DRMS: Massive Digital Resource Management System Based on OSS