Abstract:The ability to model, analyze, and predict execution time of computations is an important building block that supports numerous efforts, such as load balancing, benchmarking, job scheduling, developer-guided performance optimization, and the automation of performance tuning for high performance, parallel applications. In today’s increasingly heterogeneous computing environment, this task must be accomplished efficiently across multiple architectures, including massively parallel coprocessors like GPUs, which are increasingly prevalent in the world’s fastest supercomputers. To address this challenge, we present an approach for constructing customizable, cross-machine performance models for GPU kernels, including a mechanism to automatically and symbolically gather performance-relevant kernel operation counts, a tool for formulating mathematical models using these counts, and a customizable parameterized collection of benchmark kernels used to calibrate models to GPUs in a black-box fashion. With this approach, we empower the user to manage trade-offs between model accuracy, evaluation speed, and generalizability. A user can define their own model and customize the calibration process, making it as simple or complex as desired, and as application-targeted or general as desired. As application examples of our approach, we demonstrate both linear and nonlinear models; these examples are designed to predict execution times for multiple variants of a particular computation: two matrix-matrix multiplication variants, four discontinuous Galerkin differentiation operation variants, and two 2D five-point finite difference stencil variants. For each variant, we present accuracy results on GPUs from multiple vendors and hardware generations. We view this highly user-customizable approach as a response to a central question arising in GPU performance modeling: how can we model GPU performance in a cost-explanatory fashion while maintaining accuracy, evaluation speed, portability, and ease of use, an attribute we believe precludes approaches requiring manual collection of kernel or hardware statistics.

A quantitative roofline model for GPU kernel performance estimation using micro-benchmarks and hardware metric profiling

A quantitative performance analysis model for GPU architectures

A Unified, Hardware-Fitted, Cross-GPU Performance Model

A Simple Model for Portable and Fast Prediction of Execution Time and Power Consumption of GPU Kernels

A mechanism for balancing accuracy and scope in cross-machine black-box GPU performance modeling

Performance modeling of graphics processing unit application using static and dynamic analysis

GPGPU Performance Estimation With Core and Memory Frequency Scaling

A Performance Model for General-Purpose Computation on GPU

Quantitative GPGPU Performance Model Targeting OpenCL Architecture

OpenCL Performance Prediction using Architecture-Independent Features

A Performance Model for GPU Architectures That Considers On-Chip Resources: Application to Medical Image Registration

Hierarchical Roofline analysis for GPUs: Accelerating performance optimization for the NERSC‐9 Perlmutter system

Understanding the GPU Microarchitecture to Achieve Bare-Metal Performance Tuning

Forecasting GPU Performance for Deep Learning Training and Inference

A Performance Analysis Framework for Exploiting GPU Microarchitectural Capability.

Prediction of Performance and Power Consumption of GPGPU Applications

Evaluation of Programming Models and Performance for Stencil Computation on Current GPU Architectures

Characterizing the Execution Dynamics of GPGPU Applications

8 Steps to 3.7 TFLOP/s on NVIDIA V100 GPU: Roofline Analysis and Other Tricks

GPGPU Memory Estimation and Optimization Targeting OpenCL Architecture

An Experimental GPU Global Memory Performance Estimation and Optimization