Abstract:Modern computing paradigms, such as cloud computing, are increasingly adopting GPUs to boost their computing capabilities primarily due to the heterogeneous nature of AI/ML/deep learning workloads. However, the energy consumption of GPUs is a critical problem. Dynamic Voltage Frequency Scaling (DVFS) is a widely used technique to reduce the dynamic power of GPUs. Yet, configuring the optimal clock frequency for essential performance requirements is a non-trivial task due to the complex nonlinear relationship between the application's runtime performance characteristics, energy, and execution time. It becomes more challenging when different applications behave distinctively with similar clock settings. Simple analytical solutions and standard GPU frequency scaling heuristics fail to capture these intricacies and scale the frequencies appropriately. In this regard, we propose a data-driven frequency scaling technique by predicting the power and execution time of a given application over different clock settings. We collect the data from application profiling and train the models to predict the outcome accurately. The proposed solution is generic and can be easily extended to different kinds of workloads and GPU architectures. Furthermore, using this frequency scaling by prediction models, we present a deadline-aware application scheduling algorithm to reduce energy consumption while simultaneously meeting their deadlines. We conduct real extensive experiments on NVIDIA GPUs using several benchmark applications. The experiment results have shown that our prediction models have high accuracy with the average RMSE values of 0.38 and 0.05 for energy and time prediction, respectively. Also, the scheduling algorithm consumes 15.07% less energy as compared to the baseline policies.

GPU-Memory Coordinated Energy Saving Approach Based on Extreme Learning Machine

A DVFS Design and Simulation Framework Using Machine Learning Models

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

GPU power prediction via ensemble machine learning for DVFS space exploration

User- and Process-Driven Dynamic Voltage and Frequency Scaling.

Energy-Aware Non-Preemptive Task Scheduling with Deadline Constraint in DVFS-Enabled Heterogeneous Clusters

DSO: A GPU Energy Efficiency Optimizer by Fusing Dynamic and Static Information

Energy-aware Task Scheduling with Deadline Constraint in DVFS-enabled Heterogeneous Clusters

The Impact of GPU DVFS on the Energy and Performance of Deep Learning: an Empirical Study

Energy Efficient Real-Time Task Scheduling on CPU-GPU Hybrid Clusters

A Virtual Multi-Channel GPU Fair Scheduling Method for Virtual Machines.

A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)

A survey and measurement study of GPU DVFS on energy conservation

Learning-Directed Dynamic Voltage and Frequency Scaling Scheme with Adjustable Performance for Single-Core and Multi-Core Embedded and Mobile Systems

Predict; Do not React for Enabling Efficient Fine Grain DVFS in GPUs

An Enhanced Dynamic Voltage Scaling Scheme for Energy-Efficient Embedded Real-Time Control Systems

A power-aware system combining static compiling and DVFS

Energy-Efficient Computation with DVFS using Deep Reinforcement Learning for Multi-Task Systems in Edge Computing

Efficient Memory Management for GPU-based Deep Learning Systems

Warp-Aware Adaptive Energy Efficiency Calibration for Multi-GPU Systems

Effects of Dynamic Voltage and Frequency Scaling on a K20 GPU