Abstract:Modern computing paradigms, such as cloud computing, are increasingly adopting GPUs to boost their computing capabilities primarily due to the heterogeneous nature of AI/ML/deep learning workloads. However, the energy consumption of GPUs is a critical problem. Dynamic Voltage Frequency Scaling (DVFS) is a widely used technique to reduce the dynamic power of GPUs. Yet, configuring the optimal clock frequency for essential performance requirements is a non-trivial task due to the complex nonlinear relationship between the application's runtime performance characteristics, energy, and execution time. It becomes more challenging when different applications behave distinctively with similar clock settings. Simple analytical solutions and standard GPU frequency scaling heuristics fail to capture these intricacies and scale the frequencies appropriately. In this regard, we propose a data-driven frequency scaling technique by predicting the power and execution time of a given application over different clock settings. We collect the data from application profiling and train the models to predict the outcome accurately. The proposed solution is generic and can be easily extended to different kinds of workloads and GPU architectures. Furthermore, using this frequency scaling by prediction models, we present a deadline-aware application scheduling algorithm to reduce energy consumption while simultaneously meeting their deadlines. We conduct real extensive experiments on NVIDIA GPUs using several benchmark applications. The experiment results have shown that our prediction models have high accuracy with the average RMSE values of 0.38 and 0.05 for energy and time prediction, respectively. Also, the scheduling algorithm consumes 15.07% less energy as compared to the baseline policies.

GPU power prediction via ensemble machine learning for DVFS space exploration

A DVFS Design and Simulation Framework Using Machine Learning Models

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

Prediction models for multi-dimensional power-performance optimization on many cores

Predict; Do not React for Enabling Efficient Fine Grain DVFS in GPUs

A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)

Energy-Aware Non-Preemptive Task Scheduling with Deadline Constraint in DVFS-Enabled Heterogeneous Clusters

Orchestrated Co-scheduling, Resource Partitioning, and Power Capping on CPU-GPU Heterogeneous Systems via Machine Learning

Energy-aware Task Scheduling with Deadline Constraint in DVFS-enabled Heterogeneous Clusters

DSO: A GPU Energy Efficiency Optimizer by Fusing Dynamic and Static Information

Prediction of Performance and Power Consumption of GPGPU Applications

HPC Application Performance Prediction with Machine Learning on New Architectures

A Simple Model for Portable and Fast Prediction of Execution Time and Power Consumption of GPU Kernels

The Impact of GPU DVFS on the Energy and Performance of Deep Learning: an Empirical Study

Energy Efficient Real-Time Task Scheduling on CPU-GPU Hybrid Clusters

Prediction of GPU Failures Under Deep Learning Workloads

An Ensemble Learning Approach for In-situ Monitoring of FPGA Dynamic Power

Effects of Dynamic Voltage and Frequency Scaling on a K20 GPU

A survey and measurement study of GPU DVFS on energy conservation

Exploring the Frontiers of Energy Efficiency using Power Management at System Scale

Learning-based power prediction for data centre operations via deep neural networks