Abstract:Recent years have seen the adoption of Machine Learning (ML) techniques to predict the performance of large-scale applications, mostly at a coarse level. In contrast, we propose to use ML techniques for performance prediction at a much finer granularity, namely at the Basic Block (BB) level, which are single entry, single exit code blocks that are used for analysis by the compilers to break down a large code into manageable pieces. We extrapolate the basic block execution counts of GPU applications and use them for predicting the performance for large input sizes from the counts of smaller input sizes. We train a Poisson Neural Network (PNN) model using random input values as well as the lowest input values of the application to learn the relationship between inputs and basic block counts. Experimental results show that the model can accurately predict the basic block execution counts of 16 GPU benchmarks. We achieve an accuracy of 93.5% in extrapolating the basic block counts for large input sets when trained on smaller input sets and an accuracy of 97.7% in predicting basic block counts on random instances. In a case study, we apply the ML model to CUDA GPU benchmarks for performance prediction across a spectrum of applications. We use a variety of metrics for evaluation, including global memory requests and the active cycles of tensor cores, ALU, and FMA units. Results demonstrate the model's capability of predicting the performance of large datasets with an average error rate of 0.85% and 0.17% for global and shared memory requests, respectively. Additionally, to address the utilization of the main functional units in Ampere architecture GPUs, we calculate the active cycles for tensor cores, ALU, FMA, and FP64 units and achieve an average error of 2.3% and 10.66% for ALU and FMA units while the maximum observed error across all tested applications and units reaches 18.5%.

Machine Learning-enabled Performance Model for DNN Applications and AI Accelerator

DaDianNao: A Machine-Learning Supercomputer

Automatic Generation of Fast and Accurate Performance Models for Deep Neural Network Accelerators

A Heterogeneous Full-stack AI Platform for Performance Monitoring and Hardware-specific Optimizations

Analysis of Performance and Optimization in MindSpore on Ascend NPUs

Woodpecker-DL: Accelerating Deep Neural Networks via Hardware-Aware Multifaceted Optimizations

Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models

A Performance Analysis Framework for Exploiting GPU Microarchitectural Capability.

Performance Evaluation of MindSpore and PyTorch Based on Ascend NPU

DNN-Chip Predictor: An Analytical Performance Predictor for DNN Accelerators with Various Dataflows and Hardware Architectures

Evaluating Emerging AI/ML Accelerators: IPU, RDU, and NVIDIA/AMD GPUs

Benchmarking the Performance and Power of AI Accelerators for AI Training

Towards Universal Performance Modeling for Machine Learning Training on Multi-GPU Platforms

Performance Modeling and Evaluation of Distributed Deep Learning Frameworks on GPUs

Performance Analysis of DNN Inference/Training with Convolution and non-Convolution Operations

BB-ML: Basic Block Performance Prediction using Machine Learning Techniques

ATFormer: A Learned Performance Model with Transfer Learning Across Devices for Deep Learning Tensor Programs

Benchmarking the Performance and Energy Efficiency of AI Accelerators for AI Training

CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs

PuDianNao: A Polyvalent Machine Learning Accelerator