Abstract:The surging demand for cloud computing resources, driven by the rapid growth of sophisticated large-scale models and data centers, underscores the critical importance of efficient and adaptive resource allocation. As major tech enterprises deploy massive infrastructures with thousands of GPUs, existing cloud platforms still struggle with low resource utilization due to key challenges: capturing hierarchical indicator structures, modeling non-Gaussian distributions, and decision-making under uncertainty. To address these challenges, we propose HRAMONY, an adaptive Hierarchical Attention-based Resource Modeling and Decision-Making System. HARMONY combines hierarchical multi-indicator distribution forecasting and uncertainty-aware Bayesian decision-making. It introduces a novel hierarchical attention mechanism that comprehensively models complex inter-indicator dependencies, enabling accurate predictions that can adapt to evolving environment states. By transforming Gaussian projections into adaptive non-Gaussian distributions via Normalizing Flows. Crucially, HARMONY leverages the full predictive distributions in an adaptive Bayesian process, proactively incorporating uncertainties to optimize resource allocation while robustly meeting SLA constraints under varying conditions. Extensive evaluations across four large-scale cloud datasets demonstrate HARMONY's state-of-the-art performance, significantly outperforming nine established methods. A month-long real-world deployment validated HARMONY's substantial practical impact, realizing over 35,000 GPU hours in savings and translating to $100K+ in cost reduction, showcasing its remarkable economic value through adaptive, uncertainty-aware scaling. Our code is available at <a class="link-external link-https" href="https://github.com/Floating-LY/HARMONY1" rel="external noopener nofollow">this https URL</a>.

A Dynamic, Hierarchical Resource Model for Converged Computing

Dynamic Resource Management in a HPC and Cloud Hybrid Environment

A Deep Neural Network Based Resource Configuration Framework for Human-Machine Computing System

An Integrated Dynamic Resource Scheduling Framework in On-Demand Clouds.

An Integrated Dynamic Resource Scheduling Framework in On-Demand Clouds

ROME: A Multi-Resource Job Scheduling Framework for Exascale HPC Systems

The Flux Operator

HPC Alongside User-space Kubernetes

Towards Standard Kubernetes Scheduling Interfaces for Converged Computing

Scalable Resource Management System for High Productive Computing

Adaptive Two-Stage Cloud Resource Scaling via Hierarchical Multi-Indicator Forecasting and Bayesian Decision-Making

A Resource Representing Framework Enabling Dynamic Resource Discovery and Access Optimization.

On the Convergence of Malleability and the HPC PowerStack: Exploiting Dynamism in Over-Provisioned and Power-Constrained HPC Systems

Dynamic Resource Manager for Automating Deployments in the Computing Continuum

DYNAMIC RESOURCE ALLOCATION ENERGY-EFFICIENT FRAMEWORK FOR GREEN CLOUD COMPUTING

A Model-Driven Framework For Interoperable Cloud Resources Management

Runtime Model Based Approach to Managing Diverse Cloud Resources

Intelligent colocation of HPC workloads

Achieving better Resource Utilization by Implementing a High-Performance Intelligent Framework in a Distributed Environment

Online Resource Management in Thermal and Energy Constrained Heterogeneous High Performance Computing

Multidimensional hierarchical VM migration management for HPC cloud environments