Abstract:Modern computing paradigms, such as cloud computing, are increasingly adopting GPUs to boost their computing capabilities primarily due to the heterogeneous nature of AI/ML/deep learning workloads. However, the energy consumption of GPUs is a critical problem. Dynamic Voltage Frequency Scaling (DVFS) is a widely used technique to reduce the dynamic power of GPUs. Yet, configuring the optimal clock frequency for essential performance requirements is a non-trivial task due to the complex nonlinear relationship between the application's runtime performance characteristics, energy, and execution time. It becomes more challenging when different applications behave distinctively with similar clock settings. Simple analytical solutions and standard GPU frequency scaling heuristics fail to capture these intricacies and scale the frequencies appropriately. In this regard, we propose a data-driven frequency scaling technique by predicting the power and execution time of a given application over different clock settings. We collect the data from application profiling and train the models to predict the outcome accurately. The proposed solution is generic and can be easily extended to different kinds of workloads and GPU architectures. Furthermore, using this frequency scaling by prediction models, we present a deadline-aware application scheduling algorithm to reduce energy consumption while simultaneously meeting their deadlines. We conduct real extensive experiments on NVIDIA GPUs using several benchmark applications. The experiment results have shown that our prediction models have high accuracy with the average RMSE values of 0.38 and 0.05 for energy and time prediction, respectively. Also, the scheduling algorithm consumes 15.07% less energy as compared to the baseline policies.

Warp-Aware Adaptive Energy Efficiency Calibration for Multi-GPU Systems

A design framework for processing-in-memory accelerator

Energy Efficient Real-Time Task Scheduling on CPU-GPU Hybrid Clusters

Energy-Aware Non-Preemptive Task Scheduling with Deadline Constraint in DVFS-Enabled Heterogeneous Clusters

Energy-aware Task Scheduling with Deadline Constraint in DVFS-enabled Heterogeneous Clusters

Energy-Aware Loop Parallelism Maximization for Multi-core DSP Architectures

A Generalized Framework for Energy Savings in Real-Time Multiprocessor Systems

Efficient Kernel Management on GPUs.

Optimizing Hardware Resource Partitioning and Job Allocations on Modern GPUs under Power Caps

Effects of Dynamic Voltage and Frequency Scaling on a K20 GPU

Run-Time Technique for Simultaneous Aging and Power Optimization in GPGPUs

GPU Energy optimization based on task balance scheduling

Generalized Gpu Acceleration For Applications Employing Finite-Volume Methods

Prediction models for multi-dimensional power-performance optimization on many cores

Efficient GPU Spatial-Temporal Multitasking

An Empirical-cum-Statistical Approach to Power-Performance Characterization of Concurrent GPU Kernels

A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)

Predict; Do not React for Enabling Efficient Fine Grain DVFS in GPUs

Energy Optimization of Memory Intensive Parallel workloads

Optimizing power efficiency for 3D stacked GPU-in-memory architecture

Orchestrated Co-scheduling, Resource Partitioning, and Power Capping on CPU-GPU Heterogeneous Systems via Machine Learning