Abstract:The power & energy demands of HPC machines have grown significantly. Modern exascale HPC systems require tens of megawatts of combined power for computing resources and cooling facilities at full capacity. The current energy trend is not sustainable for future HPC systems, and there is a need to work toward the energy efficiency aspect of HPC performance. Energy awareness of the HPC applications at the job level is essential for running an efficient HPC system. This work aims to develop a pipeline to provide a production-level system-wide overview of the HPC workloads' power profile while handling evolving workloads exhibiting new power trends. We developed an open-set classification model for HPC jobs based on the properties of power profiles to continuously provide a system-wide holistic view of recently completed jobs. The pipeline helps continuously monitor the job-level power usage pattern of HPC and enables us to capture the new trends in applications' power behavior. We employed a comprehensive set of techniques to generate job-level data, custom-designed feature extraction methods to extract critical features from jobs' power profiles, clustering techniques powered by generative modeling, and open-set classification for identifying job profiles into known classes or an unknown set. With extensive evaluations, we demonstrate the effectiveness of each component in our pipeline. We provide an analysis of the resulting clusters that characterize the power profile landscape of the Summit supercomputer from more than 60K jobs executed in a year. The open-set classification classifies the known data sets into known classes with high accuracy and identifies unknown data noints with over 85% accuracy.

LIKWID Monitoring Stack: A Flexible Framework Enabling Job Specific Performance monitoring for the masses

Log-based Architectures for General-Purpose Monitoring of Deployed Code

Low-level I/O Monitoring for Scientific Workflows

LLload: Simplifying Real-Time Job Monitoring for HPC Users

MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis

Monitoring the development of CFD applications on unstable HPC platforms

Power Profile Monitoring and Tracking Evolution of System-Wide HPC Workloads

Towards Advanced Monitoring for Scientific Workflows

LLload: An Easy-to-Use HPC Utilization Tool

LLAMP: Assessing Network Latency Tolerance of HPC Applications with Linear Programming

A Security Monitoring Framework For Virtualization Based HEP Infrastructures

A Generic and Extensible Framework for Monitoring Energy Consumption of OpenStack Clouds

Multiphase and Multimode Monitoring of Batch Processes Based on Density Peak Clustering and Just-in-time Learning

Real-time control and monitoring system for LIPI's Public Cluster

Kub: Enabling Elastic HPC Workloads on Containerized Environments

From Facility to Application Sensor Data: Modular, Continuous and Holistic Monitoring with DCDB

HyperQueue: Efficient and ergonomic task graphs on HPC clusters

Enhanced Batch Process Monitoring Using Just-in-time-learning Based Kernel Partial Least Squares

A Liquidity Analysis System for Large-Scale Video Streams in the Oilfield

Pipeline Collector: gathering performance data for distributed astronomical pipelines

A Multi-Level, Multi-Scale Visual Analytics Approach to Assessment of Multifidelity HPC Systems