Abstract:Recent years have seen the adoption of Machine Learning (ML) techniques to predict the performance of large-scale applications, mostly at a coarse level. In contrast, we propose to use ML techniques for performance prediction at a much finer granularity, namely at the Basic Block (BB) level, which are single entry, single exit code blocks that are used for analysis by the compilers to break down a large code into manageable pieces. We extrapolate the basic block execution counts of GPU applications and use them for predicting the performance for large input sizes from the counts of smaller input sizes. We train a Poisson Neural Network (PNN) model using random input values as well as the lowest input values of the application to learn the relationship between inputs and basic block counts. Experimental results show that the model can accurately predict the basic block execution counts of 16 GPU benchmarks. We achieve an accuracy of 93.5% in extrapolating the basic block counts for large input sets when trained on smaller input sets and an accuracy of 97.7% in predicting basic block counts on random instances. In a case study, we apply the ML model to CUDA GPU benchmarks for performance prediction across a spectrum of applications. We use a variety of metrics for evaluation, including global memory requests and the active cycles of tensor cores, ALU, and FMA units. Results demonstrate the model's capability of predicting the performance of large datasets with an average error rate of 0.85% and 0.17% for global and shared memory requests, respectively. Additionally, to address the utilization of the main functional units in Ampere architecture GPUs, we calculate the active cycles for tensor cores, ALU, FMA, and FP64 units and achieve an average error of 2.3% and 10.66% for ALU and FMA units while the maximum observed error across all tested applications and units reaches 18.5%.

vPALs: Towards Verified Performance-aware Learning System For Resource Management

PAL: A Variability-Aware Policy for Scheduling ML Workloads in GPU Clusters

PAL -- Parallel active learning for machine-learned potentials

Analytics of Longitudinal System Monitoring Data for Performance Prediction

ML-NA: A Machine Learning Based Node Performance Analyzer Utilizing Straggler Statistics

Analytical Verification of Performance of Deep Neural Network Based Time-Synchronized Distribution System State Estimation

Automated Performance Modeling of HPC Applications Using Machine Learning.

Perph: A Workload Co-location Agent with Online Performance Prediction and Resource Inference

PALM: A Efficient Performance Simulator for Tiled Accelerators with Large-scale Model Training

Visual Diagnostics of Parallel Performance in Training Large-Scale DNN Models

LLM Performance Predictors are good initializers for Architecture Search

BB-ML: Basic Block Performance Prediction using Machine Learning Techniques

- 1-Application Performance Prediction for the Management and Scheduling of Clusters and Grids

EP4DDL: addressing straggler problem in heterogeneous distributed deep learning

Semi-Dynamic Load Balancing: Efficient Distributed Learning in Non-Dedicated Environments

Towards Universal Performance Modeling for Machine Learning Training on Multi-GPU Platforms

VELTAIR: towards high-performance multi-tenant deep learning services via adaptive compilation and scheduling

Performance Prediction and Its Use in Parallel and Distributed Computing Systems

Can We Predict Performance of Large Models across Vision-Language Tasks?

Machine Learning-enabled Performance Model for DNN Applications and AI Accelerator

Adaptive Subcarrier, Parameter, and Power Allocation for Partitioned Edge Learning Over Broadband Channels